Sampling Methods

Sampling methods are systematic procedures for selecting a subset (sample) from a larger population to estimate population characteristics. The key result: properly randomized sampling allows valid statistical inference with quantifiable error, regardless of population size. Used wherever exhaustive measurement is impossible — surveys, quality control, ecology, polling, and machine learning. Foundations laid by Bowley, Neyman, and Hansen in the early-to-mid 20th century.

A sampling method is a rule for choosing observational units from a population $U = {u_{1}, \dots, u_{N}}$ so that statistics computed on the sample estimate unknown population parameters. It matters because complete enumeration (a census) is usually infeasible, costly, or destructive, yet decisions still require knowledge of the whole. The discipline divides into probability sampling, where every unit has a known nonzero selection probability (enabling unbiased estimation and confidence statements), and non-probability sampling, which is cheaper but lacks inferential guarantees. A surprising fact: for a fixed sample size $n$ , the precision of a simple random sample depends only weakly on the population size $N$ — a sample of 1,000 estimates the U.S. population about as well as it estimates a small town's, because the finite population correction $1 - n / N$ is near 1 when $N ≫ n$ .

The Core Insight

The power of sampling comes not from the sample being "representative" by appearance, but from the randomization of selection. Because each unit's inclusion probability is known by design, we can mathematically reweight the observed data to reconstruct the population total or mean without bias, and we can quantify how much our estimate would vary across hypothetical repetitions of the sampling process. Randomness in selection, not in nature, is what licenses the inference.

Formal Statement

Population, Sample, and Inclusion Probability

Let the population be a finite set of $N$ labeled units $U = {1, 2, \dots, N}$ , each carrying a value $y_{i}$ of a study variable. A sample $S \subseteq U$ is a subset, selected by a random mechanism described by a sampling design $p (\cdot)$ , where $p (S)$ is the probability of drawing the particular subset $S$ . The first-order inclusion probability of unit $i$ is

π_{i} = Pr (i \in S) = S ∋ i \sum p (S),

and the second-order inclusion probability of the pair $(i, j)$ is

π_{ij} = Pr (i \in S and j \in S) = S \supseteq {i, j} \sum p (S) .

⟳each

π_{i}

aggregates the design probabilities over all samples containing unit

i

;

π_{ii} = π_{i}

by convention.

Population Parameters and the Sample Mean

The principal targets of estimation are the population total $T$ and population mean $\overset{ˉ}{Y}$ , with population variance $S^{2}$ :

T = i = 1 \sum N y_{i}, \overset{ˉ}{Y} = \frac{1}{N} i = 1 \sum N y_{i}, S^{2} = \frac{1}{N - 1} i = 1 \sum N (y_{i} - \overset{ˉ}{Y})^{2} .

⟳

S^{2}

uses divisor

N - 1

so that the expected sample variance equals it under simple random sampling.

Horvitz–Thompson Estimator

For any probability sampling design with $π_{i} > 0$ for all $i$ , the Horvitz–Thompson (HT) estimator of the population total,

\hat{T}_{H T} = i \in S \sum \frac{y _{i}}{π _{i}},

is unbiased: $E [\hat{T}_{H T}] = T$ . Its variance is

Var (\hat{T}_{H T}) = i = 1 \sum N j = 1 \sum N (π_{ij} - π_{i} π_{j}) \frac{y _{i}}{π _{i}} \frac{y _{j}}{π _{j}} .

⟳write

\hat{T}_{H T} = \sum_{i} (y_{i} / π_{i}) 1 [i \in S]

and use

E [1 [i \in S]] = π_{i}

and

E [1 [i \in S] 1 [j \in S]] = π_{ij}

Simple Random Sampling Without Replacement: Mean and Variance

Under simple random sampling without replacement (SRSWOR) of size $n$ from $N$ , every unit has $π_{i} = n / N$ , and the sample mean $\overset{y}{ˉ} = \frac{1}{n} \sum_{i \in S} y_{i}$ is unbiased for $\overset{ˉ}{Y}$ with variance

Var (\overset{y}{ˉ}) = \frac{S ^{2}}{n} (1 - \frac{n}{N}) .

The factor $f = n / N$ is the sampling fraction and $(1 - n / N)$ is the finite population correction (FPC).

⟳expand

Var (\overset{y}{ˉ}) = \frac{1}{n ^{2}} [\sum_{i} Var (1_{i}) y_{i}^{2} + \sum_{i \neq = j} Cov (1_{i}, 1_{j}) y_{i} y_{j}]

using

π_{i} = n / N

π_{ij} = \frac{n ( n - 1 )}{N ( N - 1 )}

Why It Holds / Derivation

Unbiasedness and Variance of the Sample Mean under SRSWOR

Step 1 — Indicator representation. Define the inclusion indicator

a_{i} = 1 [i \in S]

. Then

\overset{y}{ˉ} = \frac{1}{n} \sum_{i = 1}^{N} a_{i} y_{i}

, where the sum now runs over the whole population because

a_{i}

is zero outside

S

. Step 2 — Inclusion probabilities. By symmetry of SRSWOR, each unit is equally likely to be among the

n

chosen, so

E [a_{i}] = π_{i} = n / N

. For a pair, the number of size-

n

samples containing both is

(n - 2 N - 2)

, giving

π_{ij} = (n - 2 N - 2) / (n N) = \frac{n ( n - 1 )}{N ( N - 1 )}

. Step 3 — Unbiasedness.

E [\overset{y}{ˉ}] = \frac{1}{n} i = 1 \sum N E [a_{i}] y_{i} = \frac{1}{n} \cdot \frac{n}{N} i = 1 \sum N y_{i} = \overset{ˉ}{Y} .

Step 4 — Variances and covariances of indicators. Since

a_{i}

is Bernoulli,

Var (a_{i}) = π_{i} (1 - π_{i}) = \frac{n}{N} (1 - \frac{n}{N})

. The covariance is

Cov (a_{i}, a_{j}) = π_{ij} - π_{i} π_{j} = \frac{n ( n - 1 )}{N ( N - 1 )} - \frac{n ^{2}}{N ^{2}} = - \frac{n}{N} \cdot \frac{N - n}{N ^{2} ( N - 1 )} \cdot N = - \frac{1}{N - 1} \frac{n}{N} (1 - \frac{n}{N}) .

Step 5 — Assemble.

Var (\overset{y}{ˉ}) = \frac{1}{n ^{2}} i \sum y_{i}^{2} Var (a_{i}) + i \neq = j \sum y_{i} y_{j} Cov (a_{i}, a_{j}) .

Substituting and using

\sum_{i \neq = j} y_{i} y_{j} = T^{2} - \sum_{i} y_{i}^{2}

together with

\sum_{i} (y_{i} - \overset{ˉ}{Y})^{2} = (N - 1) S^{2}

, the algebra collapses to

Var (\overset{y}{ˉ}) = \frac{S ^{2}}{n} (1 - \frac{n}{N}) . ■

Jerzy Neyman | 1894–1981

A Polish-American mathematician and statistician, Neyman transformed survey sampling from a debated art into a rigorous theory. His landmark 1934 paper On the Two Different Aspects of the Representative Method established the superiority of randomized (probability) sampling over purposive selection and introduced confidence intervals and optimal allocation in stratified sampling. Working at University College London and later Berkeley, he founded one of the most influential statistics departments in the world. His "representative method" framework underlies essentially all modern survey practice.

Building Intuition

Think of estimating the average height of trees in a vast forest. You cannot measure every tree, so you wander in and measure some. If you measure only the tall, easy-to-reach ones near the road, your estimate is biased upward — no amount of arithmetic fixes a selection problem. But if you place a grid over a map and visit randomly chosen cells, then even though any single sample might happen to catch unusually tall or short trees, the procedure has no systematic lean. The FPC term $(1 - n / N)$ encodes a comforting fact: as your sample approaches the whole forest ( $n \to N$ ), uncertainty vanishes to zero, because you are converging on a census. The Horvitz–Thompson trick — dividing each observed value by its inclusion probability $π_{i}$ — is just bookkeeping: a tree sampled with probability $1/100$ "stands in" for 100 trees, so we weight it by 100 to rebuild the total.

Sampling design hierarchy: how units flow from population to estimate

Catalogue of Methods

Major Probability Sampling Designs

Simple Random Sampling (SRS): every subset of size $n$ equally likely; baseline design. With or without replacement.
Systematic Sampling: choose a random start $r \in {1, \dots, k}$ where $k = N / n$ , then take every $k$ -th unit. Inclusion probabilities $π_{i} = 1/ k = n / N$ .
Stratified Sampling: partition $U$ into $H$ homogeneous strata of sizes $N_{h}$ ; sample $n_{h}$ within each independently. The stratified mean estimator is

\overset{y}{ˉ}_{s t} = h = 1 \sum H W_{h} \overset{y}{ˉ}_{h}, W_{h} = \frac{N _{h}}{N},

with variance

Var (\overset{y}{ˉ}_{s t}) = \sum_{h} W_{h}^{2} \frac{S _{h}^{2}}{n _{h}} (1 - \frac{n _{h}}{N _{h}})

Cluster Sampling: sample whole groups (clusters) and measure all units within them; cheaper but usually less precise per unit.
Multistage Sampling: combine the above hierarchically (e.g., sample districts, then households, then persons).
Probability Proportional to Size (PPS): set $π_{i} \propto x_{i}$ for an auxiliary size measure $x_{i}$ , improving efficiency when $y_{i} \approx β x_{i}$ .

⟳equation [7] is a weighted average; its variance follows from independence across strata, so cross-stratum covariances vanish.

Neyman Optimal Allocation

For stratified sampling with fixed total sample size $n = \sum_{h} n_{h}$ , the allocation minimizing $Var (\overset{y}{ˉ}_{s t})$ is

n_{h} = n \cdot \frac{N _{h} S _{h}}{\sum _{k = 1}^{H} N _{k} S _{k}} .

Strata that are larger or more internally variable receive proportionally more sample.

⟳minimize

\sum_{h} W_{h}^{2} S_{h}^{2} / n_{h}

subject to

\sum_{h} n_{h} = n

using a Lagrange multiplier; the optimum satisfies

n_{h} \propto W_{h} S_{h} \propto N_{h} S_{h}

Worked Examples

SRSWOR Mean and Its Standard Error

Given: A town has

N = 4, 000

households. An SRSWOR of

n = 100

households records monthly water use; the sample mean is

\overset{y}{ˉ} = 18.0 m^{3}

and sample standard deviation

s = 6.0 m^{3}

. Step 1 — Point estimate of mean.

\hat{\overset{ˉ}{Y}} = \overset{y}{ˉ} = 18.0 m^{3}

. Step 2 — Estimate the variance of $\overset{y}{ˉ}$ . Using

s^{2} = 36

as an estimate of

S^{2}

Var (\overset{y}{ˉ}) = \frac{s ^{2}}{n} (1 - \frac{n}{N}) = \frac{36}{100} (1 - \frac{100}{4000}) = 0.36 \times 0.975 = 0.351.

Step 3 — Standard error.

SE (\overset{y}{ˉ}) = 0.351 = 0.592 m^{3}

. Step 4 — Estimate population total and its SE.

\hat{T} = N \overset{y}{ˉ} = 4000 \times 18.0 = 72, 000 m^{3}

;

SE (\hat{T}) = N \times SE (\overset{y}{ˉ}) = 4000 \times 0.592 = 2, 369 m^{3}

. Step 5 — 95% confidence interval for the mean (large-sample,

z = 1.96

18.0 \pm 1.96 \times 0.592 = 18.0 \pm 1.16.

\overset{ˉ}{Y} \in (16.84, 19.16) m^{3}, \hat{T} = 72, 000 \pm 4, 644 m^{3}

Neyman Allocation Across Two Strata

Given: A workforce of

N = 1, 000

splits into stratum A (managers,

N_{A} = 200

S_{A} = 12

) and stratum B (line workers,

N_{B} = 800

S_{B} = 4

), measuring annual training hours. Total budget allows

n = 100

. Step 1 — Compute the allocation weights.

N_{A} S_{A} = 200 \times 12 = 2400, N_{B} S_{B} = 800 \times 4 = 3200, sum = 5600.

Step 2 — Apply equation [8].

n_{A} = 100 \times \frac{2400}{5600} = 42.9 \approx 43, n_{B} = 100 \times \frac{3200}{5600} = 57.1 \approx 57.

Step 3 — Compare with proportional allocation (

n_{h} = n W_{h}

): proportional would give

n_{A} = 20

n_{B} = 80

. Neyman shifts effort toward the small but highly variable managers. Step 4 — Resulting stratified variance (ignoring FPC for brevity),

W_{A} = 0.2

W_{B} = 0.8

Var (\overset{y}{ˉ}_{s t}) = (0.2)^{2} \frac{144}{43} + (0.8)^{2} \frac{16}{57} = 0.04 \times 3.35 + 0.64 \times 0.281 = 0.134 + 0.180 = 0.314.

n_{A} = 43, n_{B} = 57, Var (\overset{y}{ˉ}_{s t}) \approx 0.314

Common Misconceptions

A large sample does not cure selection bias

The single most dangerous error is believing that more data fixes bad sampling. If the selection mechanism is correlated with $y$ (e.g., only volunteers respond), bias persists no matter how large $n$ grows. The 1936 Literary Digest poll surveyed 2.4 million people yet predicted Landon would defeat Roosevelt — Roosevelt won in a landslide — because the sample was drawn from car and telephone owners, who skewed wealthy during the Depression. A randomized sample of a few thousand would have been far more accurate. Precision shrinks with $n$ ; bias does not.

Systematic sampling can fail catastrophically with periodicity

Systematic sampling (every $k$ -th unit) usually behaves like SRS, but if the population list has a hidden period equal to $k$ , the design can be disastrously biased. Suppose we audit daily sales by sampling every 7th day starting on a Monday: with $k = 7$ we will only ever sample Mondays. If Monday sales systematically differ from the week's average, $\overset{y}{ˉ}$ is biased and its variance estimate (built assuming SRS-like behavior) is misleadingly small. The list order interacting with the step size — not randomness — produces the failure.

The sample variance $s^2/n$ formula assumes randomization, not "typical-looking" data

Confidence intervals are valid because the selection was randomized, which makes the estimator's sampling distribution computable. They are not justified by the sample "looking representative." Applying the formula $s^{2} / n$ to a convenience sample (e.g., the first 100 people who walked by) produces a number with the right units but no inferential meaning — the implied population is undefined and the coverage probability is unknown.

Connections

Sampling theory sits at the crossroads of probability and inference. The finite-population framework here contrasts with the infinite-population (model-based) view in Statistical Inference, where data are treated as i.i.d. draws from an assumed distribution rather than units of a fixed list. The unbiasedness and variance results rest on Expectation and Variance of indicator random variables. Large-sample confidence intervals invoke the Central Limit Theorem, adapted to without-replacement designs. Stratification and ratio estimation connect to Analysis of Variance (partitioning variability into within- and between-group components) and to Linear Regression through auxiliary-variable methods. The Monte Carlo "sampling" of distributions in computational statistics — Monte Carlo Methods and Markov Chain Monte Carlo — is a conceptual cousin: there one samples from a known distribution to estimate integrals, whereas survey sampling reconstructs an unknown finite population.

Real-World Applications

Election polling. Modern pollsters (Gallup, Pew, YouGov) use stratified and quota-corrected probability samples of ~1,000–2,000 respondents to estimate national vote shares within a few percentage points.
Official statistics. The U.S. Census Bureau's American Community Survey and the Current Population Survey use multistage stratified cluster designs to produce unemployment and income estimates between decennial censuses.
Manufacturing quality control. Acceptance sampling (e.g., ANSI/ASQ Z1.4 plans) inspects a random subset of a production lot to decide acceptance/rejection, balancing inspection cost against defect risk.
Epidemiology. Disease-prevalence surveys (e.g., NHANES) use complex multistage samples with PPS selection of geographic clusters to estimate national health metrics.
Ecology and forestry. Plot-based and line-transect sampling estimate species abundance, biomass, and biodiversity over landscapes too large to census; PPS-by-size selection of forest stands is routine.
Auditing and accounting. Financial auditors use monetary-unit (dollar-unit) sampling — a PPS design where larger transactions have higher inclusion probability — to detect overstatement.
Machine learning. Stratified train/test splits, importance sampling, and minibatch sampling in stochastic gradient descent all inherit sampling-theory logic to control bias and variance of estimates.

Historical Development

1895 — Anders Kiaer (Norway) proposes the "representative method," advocating partial enumeration over full census, sparking international debate.
1906 — Arthur Bowley gives the first rigorous treatment of random sampling and standard errors for social surveys in Britain.
1925 — The International Statistical Institute formally endorses the representative method after years of controversy.
1934 — Jerzy Neyman's foundational paper distinguishes random from purposive sampling, introduces stratified sampling theory, optimal allocation, and confidence intervals.
1936 — The Literary Digest fiasco vs. George Gallup's small scientific sample dramatizes the superiority of probability sampling to mass convenience samples.
1943 — Mahalanobis develops large-scale survey methods (and pilot/cost studies) at the Indian Statistical Institute.
1949 — Hansen and Hurwitz formalize unequal-probability (PPS) sampling and the theory of multistage designs at the U.S. Census Bureau.
1952 — Horvitz and Thompson publish their general unbiased estimator for arbitrary inclusion probabilities, unifying the field.
1977 — Leslie Kish's Survey Sampling and Cochran's Sampling Techniques (3rd ed.) become the canonical undergraduate/graduate references.

Subtleties & Limits

Nonresponse, frame errors, and the limits of design-based inference

Classical sampling theory assumes a perfect sampling frame (an accurate list of all units) and full response. In practice, frames are incomplete (undercoverage), units overlap or duplicate, and nonresponse is rarely random — those who decline often differ systematically from those who answer. The HT estimator's unbiasedness collapses when $π_{i}$ is not truly known because some units are unreachable. Adjustments — weighting classes, calibration, raking, and model-assisted estimators — partly repair this but reintroduce modeling assumptions, blurring the clean design-based / model-based distinction. With the decline of telephone landlines and the rise of nonprobability online panels, a live methodological frontier is whether well-calibrated nonprobability samples, paired with auxiliary data and statistical models, can match probability samples — a question without consensus. Estimating second-order inclusion probabilities $π_{ij}$ for complex designs, needed for valid variance estimation, also remains computationally and theoretically delicate, motivating resampling methods (jackknife, bootstrap for survey data).