{ "id": "ep0zn108m4", "slug": "nonuniform-sampling-statistics", "title": "Nonuniform sampling (statistics)", "updated": "2026-09-30", "topic_path": [ { "id": "physical", "label": "Physical world and mathematics", "api_url": "https://www.edgepedia.org/api/v1/topics/physical" }, { "id": "physical.math", "label": "Mathematics and statistics", "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math" }, { "id": "physical.math.statistics", "label": "Statistics and probability", "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math.statistics" }, { "id": "physical.math.statistics.stats_statistical_inference", "label": "Statistical inference, estimation, sampling, and testing", "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math.statistics.stats_statistical_inference" }, { "id": "physical.math.statistics.stats_statistical_inference.sampling_surveys", "label": "Sampling design and survey methodology", "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math.statistics.stats_statistical_inference.sampling_surveys" }, { "id": "physical.math.statistics.stats_statistical_inference.sampling_designs", "label": "Sampling designs and estimators", "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math.statistics.stats_statistical_inference.sampling_designs" }, { "id": "physical.math.statistics.stats_statistical_inference.sampling_designs.pps_and_unequal_probability_sampling", "label": "Probability-proportional-to-size and unequal-probability designs", "api_url": "https://www.edgepedia.org/api/v1/topics/physical.math.statistics.stats_statistical_inference.sampling_designs.pps_and_unequal_probability_sampling" } ], "geo": [ { "id": "geo.nongeo.t1946.physical.math.statistics.stats_statistical_inference", "label": "Non-geographic · 1946 to 2000: Statistical inference, estimation, sampling, and testing", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical.math.statistics.stats_statistical_inference", "path": [ { "id": "geo.nongeo", "label": "Non-geographic", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo" }, { "id": "geo.nongeo.t1946", "label": "Non-geographic · 1946 to 2000", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946" }, { "id": "geo.nongeo.t1946.physical", "label": "Physical world and mathematics", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical" }, { "id": "geo.nongeo.t1946.physical.math", "label": "Mathematics and statistics", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical.math" }, { "id": "geo.nongeo.t1946.physical.math.statistics", "label": "Statistics and probability", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical.math.statistics" }, { "id": "geo.nongeo.t1946.physical.math.statistics.stats_statistical_inference", "label": "Statistical inference, estimation, sampling, and testing", "api_url": "https://www.edgepedia.org/api/v1/geo/geo.nongeo.t1946.physical.math.statistics.stats_statistical_inference" } ] } ], "excerpt": "Nonuniform sampling, also called varying probability sampling, is a statistical design that selects units with unequal probabilities, weighted by their inclusion probabilities, to estimate totals more precisely.", "snippet": "Nonuniform sampling, also called varying probability sampling, is a statistical design that selects units with unequal probabilities, weighted by their inclusion probabilities, to estimate totals more precisely.", "node": "physical.math.statistics.stats_statistical_inference.sampling_designs.pps_and_unequal_probability_sampling", "markdown": "# Nonuniform sampling (statistics)\n\nNonuniform sampling in statistics is a sampling design in which the units of a finite population are selected with unequal probabilities, so that estimates of population totals and means can be made more precise than equal-probability designs allow. The design is specified by the inclusion probabilities, and estimation weights each observed unit by the reciprocal of its inclusion probability.\n\nThe method matters when units differ greatly in size or importance. Sampling unequal-sized clusters with equal probabilities is inefficient, and probability-proportional-to-size (PPS) sampling addresses that inefficiency directly.[1](https://encyclopediaofmath.org/wiki/Sampling_from_finite_populations) More generally, under certain circumstances more efficient estimators are obtained by assigning unequal probabilities of selection to population units, which is why the design is also called varying probability sampling.[2](https://home.iitk.ac.in/~shalab/sampling/chapter7-sampling-varying-probability-sampling.pdf) Unequal probability sampling is unproblematic as long as the inclusion probabilities are known and proper estimation formulas are used.[3](https://dickbrus.github.io/SpatialSamplingwithR/IntroProbabilitySampling.html)\n\n| Key fact | Detail |\n|---|---|\n| Defining quantities | First-order inclusion probability \\( \\pi_{i} \\) (unit \\( i \\) is included) and second-order \\( \\pi_{ij} \\) (units \\( i \\) and \\( j \\) are both included)[4](https://math.montana.edu/jobo/st446/documents/ho6.pdf) |\n| Main estimator | Horvitz–Thompson estimator \\( \\widehat{Y} = \\sum_{k \\in s} y_{k}/\\pi_{k} \\), unbiased for the population total[5](https://online.stat.psu.edu/stat506/Lesson03) • [6](https://homepages.ecs.vuw.ac.nz/~rarnold/STAT392/SampleSurveysBook/_book/probability-proportional-to-size-sampling.html) |\n| Founding papers | Neyman (1934)[7](https://doi.org/10.2307/2342192), Hansen and Hurwitz (1943)[8](https://doi.org/10.1214/aoms/1177731356), and Horvitz and Thompson (1952)[9](https://doi.org/10.1080/01621459.1952.10483446) |\n| Named variants | PPS, conditional Poisson, Sampford, Pareto, Rao–Hartley–Cochran, pivotal, and systematic πps designs[10](https://www.diva-portal.org/smash/get/diva2:216730/FULLTEXT01.pdf) |\n| Efficiency condition | PPS has small variance when the size variable \\( x \\) is roughly proportional to the target variable \\( y \\)[11](https://psai.ph/docs/publications/tps/tps_2019_68_2_2.pdf) |\n| Main failure mode | Unweighted means are biased under PPS; unknown inclusion probabilities preclude design-based inference[2](https://home.iitk.ac.in/~shalab/sampling/chapter7-sampling-varying-probability-sampling.pdf) • [3](https://dickbrus.github.io/SpatialSamplingwithR/IntroProbabilitySampling.html) |\n\n## How it works\n\nTwo numbers characterize a design at the unit level: the first-order inclusion probability \\( \\pi_{i} \\), the probability that unit \\( i \\) is included, and the second-order inclusion probability \\( \\pi_{ij} \\), the probability that units \\( i \\) and \\( j \\) are both included.[4](https://math.montana.edu/jobo/st446/documents/ho6.pdf) Designs whose first-order probabilities are unequal are called πps designs.[10](https://www.diva-portal.org/smash/get/diva2:216730/FULLTEXT01.pdf)\n\nThe Horvitz–Thompson (π) estimator of the total is \\( \\widehat{Y} = \\sum_{k \\in s} y_{k}/\\pi_{k} \\)[6](https://homepages.ecs.vuw.ac.nz/~rarnold/STAT392/SampleSurveysBook/_book/probability-proportional-to-size-sampling.html); because the sum runs over the ν distinct units in the sample, it does not depend on how many times a unit is selected.[4](https://math.montana.edu/jobo/st446/documents/ho6.pdf) It is unbiased since \\( E(I_{k}) = \\pi_{k} \\), and the design weight is \\( d_{k} = 1/\\pi_{k} \\).[10](https://www.diva-portal.org/smash/get/diva2:216730/FULLTEXT01.pdf) For fixed-size designs, its variance is written in the Sen–Yates–Grundy form, which involves the second-order probabilities \\( \\pi_{ij} \\).[10](https://www.diva-portal.org/smash/get/diva2:216730/FULLTEXT01.pdf) • [22](https://onlinelibrary.wiley.com/doi/10.1111/j.1467-842X.1984.tb01272.x)\n\nWhen sampling is with replacement, the Hansen–Hurwitz estimator applies. If unit i is drawn at each of n draws with probability \\( \\psi_{i} = X_{i}/X \\), then \\( \\widehat{Y}_{HH} = (1/n) \\sum_{k \\in s} y_{k}/\\psi_{k} \\) is unbiased with variance \\( \\mathrm{Var}(\\widehat{Y}_{HH}) = (1/n) \\sum_{i=1}^{N} (Y_{i}/\\psi_{i} - Y)^{2} \\psi_{i} \\), estimated unbiasedly by \\( (1/(n(n-1))) \\sum_{k \\in s} (y_{k}/\\psi_{k} - \\widehat{Y}_{HH})^{2} \\).[6](https://homepages.ecs.vuw.ac.nz/~rarnold/STAT392/SampleSurveysBook/_book/probability-proportional-to-size-sampling.html) The Horvitz–Thompson estimator can be used with or without replacement, whereas the Hansen–Hurwitz estimator is specific to with-replacement sampling.[5](https://online.stat.psu.edu/stat506/Lesson03) Ratio and regression (model-assisted) estimators build on the π-weights using ancillary variables; when \\( \\pi_{i} \\propto x_{i} \\) and \\( y_{i} \\approx r x_{i} \\), the [ratio estimator](https://www.edgepedia.org/ratio-estimator) is proportional to the [Horvitz–Thompson estimator](https://www.edgepedia.org/horvitz-thompson-estimator).[12](https://www.stats.ox.ac.uk/~ripley/StatMethods/Sampling.pdf)\n\n## How it is done\n\nThe practitioner first chooses a size measure \\( x_{i} \\) for each unit. Setting \\( \\pi_{i} \\propto x_{i} \\) defines pps sampling; constructing plans that achieve specific inclusion probabilities for fixed sample size \\( n > 1 \\) is somewhat tricky, and several methods exist.[12](https://www.stats.ox.ac.uk/~ripley/StatMethods/Sampling.pdf)\n\nSelection then follows one of the variant algorithms. For with-replacement PPS, the rejection method is a simpler way of drawing the sample than the cumulative method, using the size measures \\( X_{i} \\).[6](https://homepages.ecs.vuw.ac.nz/~rarnold/STAT392/SampleSurveysBook/_book/probability-proportional-to-size-sampling.html) For without-replacement designs, the Sampford algorithm draws the first unit with probabilities \\( \\pi_{i}/n \\), then \\( n-1 \\) further units with replacement with probabilities proportional to \\( \\pi_{i}/(1-\\pi_{i}) \\), accepting only samples of n distinct units; many samples are rejected, so this original implementation is slow.[13](https://www.sciencedirect.com/science/article/abs/pii/S037837580800387X) Software implements the standard designs: the R package 'sampling' provides Tillé, Midzuno, systematic, pivotal, and simple random sampling without replacement, with [Monte Carlo](https://www.edgepedia.org/monte-carlo) simulation used to study the accuracy of the Horvitz–Thompson estimator of a total.[14](https://cran.r-project.org/web/packages/sampling/vignettes/UPexamples.pdf)\n\n## Origin\n\nThe statistical foundations were laid by [Jerzy Neyman](https://www.edgepedia.org/jerzy-neyman), who in 1934 presented to the [Royal Statistical Society](https://www.edgepedia.org/royal-statistical-society) a classic paper comparing random and purposive selection; it contained a detailed methodology for inference from probability samples of finite populations, including a definition of a confidence interval in that context.[15](https://sit.stat.gov.pl/SiT/2023/3/gus_sit_2023_02_graham_kalton_probability_vs._nonprobability_sampling.pdf?v=2)\n\nUnequal probability selection entered survey practice with Morris H. Hansen and William N. Hurwitz, whose 1943 Annals paper \"On the Theory of [Sampling](https://www.edgepedia.org/sampling-statistics) from Finite Populations\" first suggested unequal probability selection of primary sampling units with probability proportional to size.[8](https://doi.org/10.1214/aoms/1177731356) A general theory of unequal probability sampling without replacement was first given by D. G. Horvitz and D. J. Thompson in \"A Generalization of Sampling Without Replacement from a Finite Universe\" ([Journal of the American Statistical Association](https://www.edgepedia.org/journal-of-the-american-statistical-association), 1952).[9](https://doi.org/10.1080/01621459.1952.10483446) • [16](https://projecteuclid.org/journalArticle/Download?urlId=10.1214%2Faoms%2F1177704564&isResultClick=False)\n\n## Variants\n\nThe Rao–Hartley–Cochran procedure, proposed by J. N. K. Rao, H. O. Hartley, and W. G. Cochran in 1962, is a simple without-replacement method whose total estimator has smaller variance than sampling with replacement, with simple calculation and exact variance estimation possible.[17](https://doi.org/10.1111/j.2517-6161.1962.tb00475.x)\n\nThree high-entropy πps designs are prominent. The conditional Poisson design is a maximum-entropy design, but it is difficult to determine sampling parameters that yield prescribed inclusion probabilities. The Sampford design yields prescribed inclusion probabilities but may be hard to sample from; it has the advantage over conditional Poisson and Pareto sampling that no parameter adjustment is needed.[10](https://www.diva-portal.org/smash/get/diva2:216730/FULLTEXT01.pdf) • [13](https://www.sciencedirect.com/science/article/abs/pii/S037837580800387X) The Pareto design makes sample selection very easy but makes determining the parameters very difficult[10](https://www.diva-portal.org/smash/get/diva2:216730/FULLTEXT01.pdf); Sampford samples can also be generated via Pareto samples.[13](https://www.sciencedirect.com/science/article/abs/pii/S037837580800387X) At the other end of the entropy range, systematic πps-sampling and the Pivot method give prescribed inclusion probabilities with very efficient implementations.[13](https://www.sciencedirect.com/science/article/abs/pii/S037837580800387X)\n\n## Applications\n\nIn survey sampling, PPS selection of primary sampling units in multi-stage designs addresses the inefficiency of sampling unequal-sized clusters with equal probabilities.[1](https://encyclopediaofmath.org/wiki/Sampling_from_finite_populations) In spatial and environmental surveying, the pivotal method selects samples well spread over the population, improving estimation when study variables are spatially structured, and it works in any number of dimensions.[18](https://onlinelibrary.wiley.com/doi/10.1111/j.1541-0420.2011.01699.x) In machine learning, nonuniform (importance) sampling of clients in federated learning can speed convergence in stochastic optimization; a JMLR paper casts client sampling as online learning with bandit feedback, solved by an online stochastic mirror descent algorithm that minimizes sampling variance, and provably improves convergence over uniform sampling.[19](https://jmlr.org/papers/volume26/24-0385/24-0385.pdf) In survey methodology, a 2025 Bayesian adaptive survey design paper frames obtaining a balanced respondent set within budget as an optimization problem that allocates data collection strategies across strata to minimize nonresponse bias, subject to response-rate and budget constraints.[20](https://sage.cnpereading.com/doi/10.1177/00491241251345463)\n\n## Limitations and alternatives\n\nThe central failure mode is estimation with the wrong weights. In PPS sampling, the unweighted sample mean is generally biased because larger units are overrepresented and smaller units underrepresented; unbiased estimation uses inverse per-draw selection probabilities in the Hansen–Hurwitz estimator for sampling with replacement, or inverse inclusion probabilities for distinct sampled units in the Horvitz–Thompson estimator.[2](https://home.iitk.ac.in/~shalab/sampling/chapter7-sampling-varying-probability-sampling.pdf) • [23](https://doi.org/10.1002/9781118445112.stat03346.pub2) • [24](https://psirusteam.github.io/HouseholdSurveysR/ap2.html) More fundamentally, design-based estimation is impossible for arbitrary or haphazard sampling because it requires the inclusion probabilities determined by the sampling design; only model-based inference is then possible.[3](https://dickbrus.github.io/SpatialSamplingwithR/IntroProbabilitySampling.html) Model-based (superpopulation) inference assumes a stochastic model for the generation of the \\( y_{j} \\) and can adjust for non-response, which design-based inference cannot.[12](https://www.stats.ox.ac.uk/~ripley/StatMethods/Sampling.pdf)\n\nOptimality claims need a correct model. The strategy coupling PPS sampling with the GREG estimator has been called \"optimal\" because it minimizes the anticipated variance, but that optimality relies on the finite population being a realization of a known superpopulation model; gross errors may be observed when a misspecified model is used.[21](http://gauss.stat.su.se/rr/RR2018_6.pdf) In comparisons, model-based stratification combined with the GREG estimator, although theoretically less efficient, has sometimes been empirically more efficient than the \"optimal\" PPS+GREG strategy.[21](http://gauss.stat.su.se/rr/RR2018_6.pdf) Against stratified sampling, unequal probability sampling is a different lever: stratification controls variance through [Neyman allocation](https://www.edgepedia.org/neyman-allocation), the allocation of a fixed overall sample size across strata that minimizes the variance of an overall estimate[1](https://encyclopediaofmath.org/wiki/Sampling_from_finite_populations), while πps designs control it through unequal selection within the whole population.\n\n## References\n\n1. [Sampling from finite populations (Encyclopedia of Mathematics)](https://encyclopediaofmath.org/wiki/Sampling_from_finite_populations)\n2. [Chapter 7: Varying Probability Sampling (IIT Kanpur)](https://home.iitk.ac.in/~shalab/sampling/chapter7-sampling-varying-probability-sampling.pdf)\n3. [Spatial sampling with R – Introduction to probability sampling](https://dickbrus.github.io/SpatialSamplingwithR/IntroProbabilitySampling.html)\n4. [Unequal Probability Sampling (STAT 446 handout)](https://math.montana.edu/jobo/st446/documents/ho6.pdf)\n5. [Unequal Probability Sampling – STAT 506 | Sampling Theory and Methods](https://online.stat.psu.edu/stat506/Lesson03)\n6. [Chapter 7 Probability Proportional to Size Sampling | STAT392: Sample Surveys](https://homepages.ecs.vuw.ac.nz/~rarnold/STAT392/SampleSurveysBook/_book/probability-proportional-to-size-sampling.html)\n7. [Jerzy Neyman (1934). On the Two Different Aspects of the Representative Method: The Method of Stratified Sampling and the Method of Purposive Selection. Journal Of The Royal Statistical Society.](https://doi.org/10.2307/2342192)\n8. [Morris H. Hansen, William N. Hurwitz (1943). On the Theory of Sampling from Finite Populations. The Annals of Mathematical Statistics.](https://doi.org/10.1214/aoms/1177731356)\n9. [D. G. Horvitz, D. J. Thompson (1952). A Generalization of Sampling Without Replacement from a Finite Universe. Journal of the American Statistical Association.](https://doi.org/10.1080/01621459.1952.10483446)\n10. [Contributions to the Theory of Unequal Probability Sampling (thesis)](https://www.diva-portal.org/smash/get/diva2:216730/FULLTEXT01.pdf)\n11. [Sampling with Probability Proportional to Aggregate Size in Heterogeneous Populations](https://psai.ph/docs/publications/tps/tps_2019_68_2_2.pdf)\n12. [Analysis of Sampling Plans (Ripley, Oxford)](https://www.stats.ox.ac.uk/~ripley/StatMethods/Sampling.pdf)\n13. [Non-rejective implementations of the Sampford sampling design](https://www.sciencedirect.com/science/article/abs/pii/S037837580800387X)\n14. [Unequal probability sampling designs (R package 'sampling' vignette)](https://cran.r-project.org/web/packages/sampling/vignettes/UPexamples.pdf)\n15. [Probability vs. Nonprobability Sampling: From the Birth of Survey Sampling to the Present Day (Kalton)](https://sit.stat.gov.pl/SiT/2023/3/gus_sit_2023_02_graham_kalton_probability_vs._nonprobability_sampling.pdf?v=2)\n16. [Paper on unequal probability sampling without replacement (Annals of Mathematical Statistics, Project Euclid)](https://projecteuclid.org/journalArticle/Download?urlId=10.1214%2Faoms%2F1177704564&isResultClick=False)\n17. [J. N. K. Rao, H. O. Hartley, W. G. Cochran (1962). On a Simple Procedure of Unequal Probability Sampling Without Replacement. Journal of the Royal Statistical Society Series B (Statistical Methodology).](https://doi.org/10.1111/j.2517-6161.1962.tb00475.x)\n18. [Spatially Balanced Sampling through the Pivotal Method (Biometrics)](https://onlinelibrary.wiley.com/doi/10.1111/j.1541-0420.2011.01699.x)\n19. [Adaptive Client Sampling in Federated Learning via Online Learning with Bandit Feedback](https://jmlr.org/papers/volume26/24-0385/24-0385.pdf)\n20. [An Optimal Stratification Method for Addressing Nonresponse Bias in Bayesian Adaptive Survey Design](https://sage.cnpereading.com/doi/10.1177/00491241251345463)\n21. [Research Report (Stockholm University, 2018)](http://gauss.stat.su.se/rr/RR2018_6.pdf)\n22. [J.1467 842X.1984.tb01272.x (onlinelibrary.wiley.com)](https://onlinelibrary.wiley.com/doi/10.1111/j.1467-842X.1984.tb01272.x)\n23. [doi.org](https://doi.org/10.1002/9781118445112.stat03346.pub2)\n24. [Appendix B Finite Population Inference, Household Surveys Analysis with R](https://psirusteam.github.io/HouseholdSurveysR/ap2.html)\n\n---\n*Topic: Encyclopedia › Physical world and mathematics › Mathematics and statistics › Statistics and probability › Statistical inference, estimation, sampling, and testing › Sampling design and survey methodology › Sampling designs and estimators › Probability-proportional-to-size and unequal-probability designs*\n\n*Initially written Sep 29, 2026 · Reviewed: Sep 30, 2026 · Edited: Sep 30, 2026 · Last review: Sep 30, 2026*\n\n*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*\n\nLicense: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license\n", "same_as": [], "url": "https://www.edgepedia.org/nonuniform-sampling-statistics", "markdown_url": "https://www.edgepedia.org/nonuniform-sampling-statistics.md", "license": { "name": "Edgepedia Community License 1.0", "url": "https://www.edgepedia.org/edgepedia/license", "summary": "Free with credit, commercial use included. AI training is open to everyone. For other uses, organizations over USD 100M in revenue or 100M monthly users license separately.", "spdx": "LicenseRef-Edgepedia-Community-1.0" }, "credit": "\"Nonuniform sampling (statistics)\", Edgepedia (EdgeChat), https://www.edgepedia.org/nonuniform-sampling-statistics. Edgepedia Community License 1.0.", "credit_md": "\"[Nonuniform sampling (statistics)](https://www.edgepedia.org/nonuniform-sampling-statistics)\", Edgepedia (EdgeChat), [https://www.edgepedia.org/nonuniform-sampling-statistics](https://www.edgepedia.org/nonuniform-sampling-statistics). [Edgepedia Community License 1.0](https://www.edgepedia.org/edgepedia/license).", "credit_html": "\"Nonuniform sampling (statistics)\", Edgepedia (EdgeChat), https://www.edgepedia.org/nonuniform-sampling-statistics. Edgepedia Community License 1.0.", "speakable": "Nonuniform sampling, also called varying probability sampling, is a statistical design that selects units with unequal probabilities, weighted by their inclusion probabilities, to estimate totals more precisely." }