Sampling Distributions

Suppose you weigh 30 apples from an orchard and compute their average weight. Repeat the experiment tomorrow with a fresh batch of 30 apples. The average will come out slightly different. If you could repeat this sampling forever, the collection of all these averages would trace out its own probability distribution. This is the sampling distribution of the sample mean. This idea is the hinge on which all of statistical inference turns. A statistic computed from data is itself a random variable. To say anything rigorous about how much we trust it, we must know how it behaves. Specifically, we need to know its behavior across hypothetical repetitions of the sampling process. The importance of this idea is easy to underrate. It looks almost tautological once stated. But it resolves a genuine conceptual puzzle. How can a single number, computed from one particular sample, tell us something about an unknown population parameter? And how can it come with a quantified sense of how reliable that number is? The answer is that the single number is one realized value of a random variable. We can, in favorable cases, work out its full distribution exactly or approximately. That distribution encodes exactly the reliability information we need. This article develops the sampling distribution concept rigorously. We explain what it means for a statistic to have a distribution "induced" by that of the underlying random sample. We show how to compute such distributions exactly in tractable cases (using Distributions of Transformed Random Variables). We also examine how two central limit phenomena govern the typical behavior of sample means. These are exact normality under a normal population, and asymptotic normality in general via the Central Limit Theorem. We build everything from the definition of a random sample. We prove the key convolution and normality facts from first principles, rather than asserting them. By the end, the reader should be equipped to state precisely what a sampling distribution is. The reader should be able to derive it in worked examples with different sample sizes and distributions. The reader should recognize the specific hypotheses (independence, finite variance, sample size) that make each theorem work. Finally, the reader should see how this concept feeds directly into The Sample Mean and Sample Variance, Confidence Intervals: Concept and Construction, and The t, Chi-Squared, and F Distributions.

Key takeaways
  • A statistic T=g(X1,…,Xn)T = g(X_1,\dots,X_n) computed from a random sample is itself a random variable; its distribution is the sampling distribution of TT.
  • Sampling distributions depend on the population distribution, the sample size nn, and the functional form of the statistic gg.
  • If X1,…,XnX_1,\dots,X_n are i.i.d. N(μ,σ2)N(\mu,\sigma^2), then exactly, for every finite nn: Xˉn∼N(μ,σ2/n)\bar X_n \sim N(\mu, \sigma^2/n).
  • If the XiX_i are i.i.d. with finite mean μ\mu and variance σ2\sigma^2 (any distribution), the Central Limit Theorem gives n(Xˉn−μ)/σ→dN(0,1)\sqrt n(\bar X_n - \mu)/\sigma \xrightarrow{d} N(0,1) as n→∞n\to\infty.
  • Sampling distributions always have mean equal to the target functional (when unbiased) and a spread that shrinks like 1/n1/\sqrt n for the sample mean — this is the mathematical content of "more data means more precision."
  • Key idea: inference works because a statistic's own randomness, across hypothetical repeated sampling, is itself something we can compute or approximate.

Where this lives today

Sampling distributions are the computational backbone of nearly every quantitative discipline that uses data. In manufacturing quality control, control charts (e.g. Xˉ\bar X-charts) plot sample means against control limits derived exactly from the theorem above, flagging a process as "out of control" when a sample mean falls in the extreme tail of its sampling distribution. In clinical trials and biostatistics, the sampling distribution of a treatment-effect estimator underlies every reported confidence interval and p-value — see Confidence Intervals: Concept and Construction for the direct next step. In survey sampling and polling, the "margin of error" reported alongside a poll result is literally two standard deviations of the sampling distribution of a sample proportion, itself a special case of the sample mean applied to Bernoulli data. In computational statistics, the bootstrap approximates an intractable sampling distribution by resampling from the observed data, substituting simulation for the closed-form derivations shown here when exact or asymptotic formulas are unavailable. In physics and metrology, propagation of measurement uncertainty through repeated trials rests on exactly the variance-scaling law σ2/n\sigma^2/n derived in part (a). And in machine learning, the sampling distribution of an estimator (e.g. a regression coefficient) underlies statistical learning theory bounds on generalization error, connecting directly to The t, Chi-Squared, and F Distributions used to build exact tests when the population variance is unknown.

Ask the tutor: Why does the sampling distribution of the sample mean narrow at rate 1/n1/\sqrt n specifically, rather than 1/n1/n or some other rate?

DefinitionSampling Distribution

Let X1,X2,…,XnX_1, X_2, \dots, X_n be a random sample from a population with distribution FF — that is, i.i.d.\ random variables each with cumulative distribution function FF. Let

T=g(X1,…,Xn)T = g(X_1, \dots, X_n)

be a statistic: a Borel-measurable function of the sample that does not depend on any unknown parameter of FF. Since TT is a measurable function of random variables, TT is itself a random variable, and it therefore possesses a cumulative distribution function

FT(t)=P(T≤t),t∈R.F_T(t) = P(T \le t), \qquad t \in \mathbb{R}.

This distribution FTF_T — equivalently the induced probability measure PT=P∘T−1P_T = P \circ T^{-1} on (R,B)(\mathbb{R}, \mathcal{B}) — is called the sampling distribution of TT. The sampling distribution depends in general on three ingredients: the population law FF, the sample size nn, and the functional form gg. Two statistics computed from the same sample (e.g., the sample mean and the sample maximum) generally have different sampling distributions.

Intuition

Think of the population as an urn (possibly infinite) from which we draw nn balls, each labeled with a number, independently and with the same underlying law each time. A statistic is a rule for turning the nn drawn numbers into a single number — an average, a maximum, a variance, anything measurable. Now imagine performing the entire draw-and-compute procedure over and over, an unlimited number of times, recording the resulting number each time. The relative frequency histogram of these numbers, in the limit of infinitely many repetitions, is the sampling distribution. The crucial conceptual shift from "a statistic" to "the sampling distribution of a statistic" is the shift from a single realized number (your one sample mean, xˉ=24.3\bar x = 24.3 grams) to the entire probabilistic machine that could have produced it. Once we know that machine — its shape, its center, its spread — we can say things like "an observed sample mean this large would occur by chance alone less than 5% of the time if μ\mu were really 20," which is the seed of hypothesis testing and confidence intervals. The two figures below illustrate this for the sample mean: first, how repeated sampling produces a cloud of possible Xˉn\bar X_n values concentrating around μ\mu; second, how the shape of that cloud sharpens from a skewed population distribution toward a bell curve as nn grows — the qualitative content of the Central Limit Theorem.

Figure
Repeated sampling produces a distribution of the sample mean, centered at μ with spread σ/√n
value of X̄ₙ μ σ/√n spread each dot = one sample's X̄ₙ
Figure
As n grows, the sampling distribution of X̄ₙ becomes more sharply bell-shaped around μ (CLT)
X̄ₙ n = 4 (wide, less smooth) n = 25 n = 100 (narrower, more bell-shaped)
TheoremExact and Asymptotic Sampling Distribution of the Sample Mean

Let X1,…,XnX_1, \dots, X_n be a random sample (i.i.d.) with common mean E[Xi]=μE[X_i] = \mu and common finite variance Var(Xi)=σ2<∞\mathrm{Var}(X_i) = \sigma^2 < \infty. Let

Xˉn=1n∑i=1nXi.\bar X_n = \frac{1}{n}\sum_{i=1}^n X_i.

(a) Exact moments (any population, finite variance). For every n≥1n \ge 1,

E[Xˉn]=μ,Var(Xˉn)=σ2n.E[\bar X_n] = \mu, \qquad \mathrm{Var}(\bar X_n) = \frac{\sigma^2}{n}.

(b) Exact normality (normal population). If in addition Xi∼N(μ,σ2)X_i \sim N(\mu, \sigma^2) for each ii, then for every finite n≥1n \ge 1,

Xˉn∼N ⁣(μ,σ2n)\bar X_n \sim N\!\left(\mu, \frac{\sigma^2}{n}\right)

exactly.

(c) Asymptotic normality (Central Limit Theorem, general population). For an arbitrary population with finite mean μ\mu and finite positive variance σ2\sigma^2 (no further assumption on the shape of FF), define the standardized statistic

Zn=Xˉn−μσ/n=∑i=1nXi−nμσn.Z_n = \frac{\bar X_n - \mu}{\sigma/\sqrt n} = \frac{\sum_{i=1}^n X_i - n\mu}{\sigma \sqrt n}.

Then as n→∞n \to \infty,

Zn→dZ∼N(0,1),Z_n \xrightarrow{d} Z \sim N(0,1),

that is, P(Zn≤z)→Φ(z)P(Z_n \le z) \to \Phi(z) for every z∈Rz \in \mathbb{R}, where Φ\Phi is the standard normal c.d.f.

Proof

We prove parts (a), (b), (c) in turn. Throughout, X1,…,XnX_1,\dots,X_n are independent and identically distributed.

Proof of (a). By linearity of expectation (valid for any finite collection of random variables with finite means, with no independence needed),

E[Xˉn]=1n∑i=1nE[Xi]=1n(nμ)=μ.E[\bar X_n] = \frac{1}{n}\sum_{i=1}^n E[X_i] = \frac{1}{n}(n\mu) = \mu.

For the variance, since the XiX_i are independent, covariances between distinct terms vanish: Cov(Xi,Xj)=0\mathrm{Cov}(X_i,X_j) = 0 for i≠ji \ne j. Hence

Var(Xˉn)=Var(1n∑i=1nXi)=1n2Var(∑i=1nXi)=1n2(∑i=1nVar(Xi)+∑i≠jCov(Xi,Xj))=1n2(nσ2)=σ2n.\mathrm{Var}(\bar X_n) = \mathrm{Var}\left(\frac{1}{n}\sum_{i=1}^n X_i\right) = \frac{1}{n^2}\mathrm{Var}\left(\sum_{i=1}^n X_i\right) = \frac{1}{n^2}\left(\sum_{i=1}^n \mathrm{Var}(X_i) + \sum_{i \ne j}\mathrm{Cov}(X_i,X_j)\right) = \frac{1}{n^2}(n\sigma^2) = \frac{\sigma^2}{n}.

This uses only independence (for the covariance terms to vanish) and finiteness of σ2\sigma^2; it requires no assumption on the shape of the population distribution.

Proof of (b). We use moment generating functions (MGFs), which is legitimate here because the normal distribution's MGF exists in a neighborhood of 00 and MGFs, when they exist on a common open interval around 00, uniquely determine the distribution (this is a standard fact from the theory of transforms — it follows from uniqueness of the bilateral Laplace/MGF transform on distributions with all moments finite). If X∼N(μ,σ2)X \sim N(\mu,\sigma^2), its MGF is

MX(t)=E[etX]=exp⁡(μt+σ2t22),t∈R.M_X(t) = E[e^{tX}] = \exp\left(\mu t + \frac{\sigma^2 t^2}{2}\right), \qquad t \in \mathbb{R}.

(This is obtained by direct computation of ∫−∞∞etx12πσe−(x−μ)2/(2σ2) dx\int_{-\infty}^\infty e^{tx}\frac{1}{\sqrt{2\pi}\sigma}e^{-(x-\mu)^2/(2\sigma^2)}\,dx, completing the square in the exponent.) Since X1,…,XnX_1,\dots,X_n are independent, the MGF of the sum Sn=∑i=1nXiS_n = \sum_{i=1}^n X_i factors:

MSn(t)=E ⁣[et∑iXi]=E ⁣[∏i=1netXi]=∏i=1nE[etXi]=(exp⁡(μt+σ2t22))n=exp⁡(nμt+nσ2t22),M_{S_n}(t) = E\!\left[e^{t\sum_i X_i}\right] = E\!\left[\prod_{i=1}^n e^{tX_i}\right] = \prod_{i=1}^n E[e^{tX_i}] = \left(\exp\left(\mu t + \frac{\sigma^2t^2}{2}\right)\right)^n = \exp\left(n\mu t + \frac{n\sigma^2 t^2}{2}\right),

where the middle equality (expectation of a product of independent random variables equals the product of expectations) is a standard consequence of independence. Thus Sn∼N(nμ,nσ2)S_n \sim N(n\mu, n\sigma^2) by the uniqueness-of-MGF fact, since exp⁡(nμt+nσ2t2/2)\exp(n\mu t + n\sigma^2 t^2/2) is exactly the MGF of a N(nμ,nσ2)N(n\mu, n\sigma^2) random variable. Now Xˉn=Sn/n\bar X_n = S_n/n is a linear (scalar) transformation of SnS_n. For any random variable YY with MGF MYM_Y and constant c≠0c \ne 0, the MGF of cYcY is McY(t)=E[etcY]=MY(ct)M_{cY}(t) = E[e^{tcY}] = M_Y(ct). Applying this with Y=SnY = S_n, c=1/nc = 1/n:

MXˉn(t)=MSn(t/n)=exp⁡(nμ⋅tn+nσ22⋅t2n2)=exp⁡(μt+(σ2/n)t22).M_{\bar X_n}(t) = M_{S_n}(t/n) = \exp\left(n\mu \cdot \frac{t}{n} + \frac{n\sigma^2}{2}\cdot\frac{t^2}{n^2}\right) = \exp\left(\mu t + \frac{(\sigma^2/n) t^2}{2}\right).

This is precisely the MGF of a N(μ,σ2/n)N(\mu, \sigma^2/n) distribution. By uniqueness of MGFs, Xˉn∼N(μ,σ2/n)\bar X_n \sim N(\mu, \sigma^2/n) exactly, for every finite nn. ■\blacksquare (part b)

Proof of (c). This is the Lindeberg–Lévy Central Limit Theorem specialized to i.i.d. summands; we give a full proof via characteristic functions, which (unlike MGFs) always exist for any distribution with finite mean, since ∣eitX∣=1|e^{itX}| = 1 makes E[eitX]E[e^{itX}] absolutely convergent for every real tt. Let Yi=(Xi−μ)/σY_i = (X_i - \mu)/\sigma, so that E[Yi]=0E[Y_i] = 0 and Var(Yi)=1\mathrm{Var}(Y_i) = 1, and Y1,…,YnY_1,\dots,Y_n are still i.i.d. Let φ(t)=E[eitY1]\varphi(t) = E[e^{itY_1}] be the common characteristic function of the YiY_i. Note Zn=1n∑i=1nYiZ_n = \frac{1}{\sqrt n}\sum_{i=1}^n Y_i. Step 1: Taylor expansion of φ\varphi near 00. Since E[Y1]=0E[Y_1] = 0 and E[Y12]=1<∞E[Y_1^2] = 1 < \infty, the characteristic function is twice continuously differentiable at 00 with φ(0)=1\varphi(0) = 1, φ′(0)=iE[Y1]=0\varphi'(0) = iE[Y_1] = 0, φ′′(0)=i2E[Y12]=−1\varphi''(0) = i^2 E[Y_1^2] = -1. By Taylor's theorem with Peano remainder (valid because the second derivative exists and is continuous at 00, which holds whenever the second moment is finite — a standard fact about characteristic functions),

φ(t)=1−t22+o(t2)as t→0.\varphi(t) = 1 - \frac{t^2}{2} + o(t^2) \quad \text{as } t \to 0.

Step 2: Characteristic function of ZnZ_n. By independence of the YiY_i, the characteristic function of a sum of independent random variables is the product of the individual characteristic functions (proved exactly as the MGF factorization above, replacing etXe^{tX} with eitXe^{itX}). Hence

φZn(t)=E ⁣[exp⁡(it⋅1n∑i=1nYi)]=∏i=1nE ⁣[exp⁡(itnYi)]=[φ ⁣(tn)]n.\varphi_{Z_n}(t) = E\!\left[\exp\left(it \cdot \frac{1}{\sqrt n}\sum_{i=1}^n Y_i\right)\right] = \prod_{i=1}^n E\!\left[\exp\left(i \frac{t}{\sqrt n} Y_i\right)\right] = \left[\varphi\!\left(\frac{t}{\sqrt n}\right)\right]^n.

Step 3: Take the limit. Fix t∈Rt \in \mathbb{R}. As n→∞n \to \infty, t/n→0t/\sqrt n \to 0, so by Step 1,

φ ⁣(tn)=1−t22n+o ⁣(1n).\varphi\!\left(\frac{t}{\sqrt n}\right) = 1 - \frac{t^2}{2n} + o\!\left(\frac{1}{n}\right).

Therefore

φZn(t)=(1−t22n+o ⁣(1n))n.\varphi_{Z_n}(t) = \left(1 - \frac{t^2}{2n} + o\!\left(\frac1n\right)\right)^n.

Using the standard limit (1+ann)n→ea\left(1 + \frac{a_n}{n}\right)^n \to e^{a} whenever an→aa_n \to a (a consequence of taking logarithms and applying log⁡(1+x)=x+o(x)\log(1+x) = x + o(x) as x→0x \to 0), with an=−t2/2+n⋅o(1/n)→−t2/2a_n = -t^2/2 + n \cdot o(1/n) \to -t^2/2, we obtain

φZn(t)⟶e−t2/2for every fixed t∈R.\varphi_{Z_n}(t) \longrightarrow e^{-t^2/2} \qquad \text{for every fixed } t \in \mathbb{R}.

Step 4: Identify the limit and conclude convergence in distribution. The function e−t2/2e^{-t^2/2} is exactly the characteristic function of the standard normal distribution N(0,1)N(0,1). By Lévy's continuity theorem (a standard result: pointwise convergence of characteristic functions to a function continuous at 00 implies convergence in distribution of the corresponding random variables), we conclude

Zn→dN(0,1).Z_n \xrightarrow{d} N(0,1).

Finally, note Zn=1n∑iXi−μσ=Xˉn−μσ/nZ_n = \frac{1}{\sqrt n}\sum_i \frac{X_i - \mu}{\sigma} = \frac{\bar X_n - \mu}{\sigma/\sqrt n} by direct algebra, which is exactly the standardized statistic in the theorem statement. ■\blacksquare (part c) No step in parts (a)–(c) invokes the conclusion being proved or any result built on top of it; MGF/characteristic-function algebra, Taylor's theorem, and Lévy continuity are independent, prior tools.

∎
CorollaryApproximate sampling distribution for large n

Under the hypotheses of part (c), for large nn the sampling distribution of Xˉn\bar X_n is well approximated by

Xˉn ∼˙ N ⁣(μ,σ2n),\bar X_n \ \dot\sim\ N\!\left(\mu, \frac{\sigma^2}{n}\right),

in the sense that P(Xˉn≤x)≈Φ ⁣(x−μσ/n)P(\bar X_n \le x) \approx \Phi\!\left(\frac{x - \mu}{\sigma/\sqrt n}\right), with the approximation improving as n→∞n \to \infty. This holds regardless of the shape of the original population distribution, as long as σ2<∞\sigma^2 < \infty.

Worked examples

Example 1 (Exact sampling distribution, non-normal population, small n). A component's lifetime (in hundreds of hours) is modeled as X∼Exponential(λ=1/3)X \sim \mathrm{Exponential}(\lambda = 1/3), so E[X]=3E[X] = 3, Var(X)=9\mathrm{Var}(X) = 9. Suppose n=4n = 4 components are tested independently and we form the total lifetime S4=X1+X2+X3+X4S_4 = X_1 + X_2 + X_3 + X_4. Find the exact sampling distribution of S4S_4, and use it to compute P(S4>15)P(S_4 > 15). Solution. The sum of nn i.i.d. Exponential(λ)\mathrm{Exponential}(\lambda) random variables is Gamma(n,λ)\mathrm{Gamma}(n, \lambda) (shape nn, rate λ\lambda) — this is a standard convolution fact provable by induction using the MGF: MX(t)=λ/(λ−t)M_X(t) = \lambda/(\lambda - t) for t<λt < \lambda, so by independence

MS4(t)=(λλ−t)4,M_{S_4}(t) = \left(\frac{\lambda}{\lambda - t}\right)^4,

which is the MGF of Gamma(4,λ)\mathrm{Gamma}(4,\lambda), and MGF uniqueness identifies the distribution. With λ=1/3\lambda = 1/3, S4∼Gamma(4,1/3)S_4 \sim \mathrm{Gamma}(4, 1/3), with density

fS4(s)=λ43!s3e−λs=(1/3)46s3e−s/3,s>0.f_{S_4}(s) = \frac{\lambda^4}{3!}s^3 e^{-\lambda s} = \frac{(1/3)^4}{6}s^3 e^{-s/3}, \qquad s > 0.

This is the sampling distribution of the statistic S4S_4: it is exact for this n=4n=4, not an approximation. To find P(S4>15)P(S_4 > 15), we use the well-known relation between the Gamma(k,λ)(k,\lambda) c.d.f. (integer shape) and a Poisson tail: if S4∼Gamma(4,λ)S_4 \sim \mathrm{Gamma}(4,\lambda) then

P(S4>s)=∑j=03(λs)jj!e−λsP(S_4 > s) = \sum_{j=0}^{3} \frac{(\lambda s)^j}{j!}e^{-\lambda s}

(this identity follows from repeated integration by parts on the Gamma density, or equivalently from viewing S4S_4 as the waiting time of the 4th event in a Poisson process of rate λ\lambda, so {S4>s}={N(s)≤3}\{S_4 > s\} = \{N(s) \le 3\} where N(s)∼Poisson(λs)N(s)\sim\mathrm{Poisson}(\lambda s)). With λs=(1/3)(15)=5\lambda s = (1/3)(15) = 5:

P(S4>15)=e−5(1+5+252+1256)=e−5(1+5+12.5+20.83‾)=e−5(39.33‾).P(S_4 > 15) = e^{-5}\left(1 + 5 + \frac{25}{2} + \frac{125}{6}\right) = e^{-5}\left(1 + 5 + 12.5 + 20.8\overline{3}\right) = e^{-5}(39.3\overline{3}).

Numerically e−5≈0.0067379e^{-5} \approx 0.0067379, so

P(S4>15)≈0.0067379×39.3333≈0.2650.P(S_4 > 15) \approx 0.0067379 \times 39.3333 \approx 0.2650.

Cross-check with the CLT (illustrating why n=4 is genuinely "small"). Treating S4S_4 as approximately normal via part (c) applied to the sum (mean nμ=12n\mu = 12, variance nσ2=36n\sigma^2 = 36, so sd 66):

P(S4>15)≈P ⁣(Z>15−126)=P(Z>0.5)=1−Φ(0.5)≈1−0.6915=0.3085.P(S_4 > 15) \approx P\!\left(Z > \frac{15-12}{6}\right) = P(Z > 0.5) = 1 - \Phi(0.5) \approx 1 - 0.6915 = 0.3085.

The normal approximation (0.3085) differs noticeably from the exact value (0.2650) precisely because n=4n=4 is small and the exponential population is heavily right-skewed — a concrete illustration that the CLT is an asymptotic statement, and the exact Gamma sampling distribution is the correct tool here.

Example 2 (Exact normal sampling distribution, unequal-looking numbers). Cartons of orange juice are filled by a machine with fill volumes X∼N(502,6.25)X \sim N(502, 6.25) (mL; so σ=2.5\sigma = 2.5 mL, a slight positive overfill bias against a nominal 500 mL label). A quality inspector samples n=16n = 16 cartons at random and computes the sample mean fill Xˉ16\bar X_{16}. Find P(500≤Xˉ16≤503)P(500 \le \bar X_{16} \le 503), and find the value cc such that P(Xˉ16>c)=0.01P(\bar X_{16} > c) = 0.01. Solution. By part (b) of the theorem (exact, since the population is normal), for every nn,

Xˉ16∼N ⁣(502,6.2516)=N(502, 0.390625).\bar X_{16} \sim N\!\left(502, \frac{6.25}{16}\right) = N(502,\ 0.390625).

So the standard deviation of Xˉ16\bar X_{16} is 0.390625=0.625\sqrt{0.390625} = 0.625 mL. Standardizing:

P(500≤Xˉ16≤503)=P ⁣(500−5020.625≤Z≤503−5020.625)=P(−3.2≤Z≤1.6).P(500 \le \bar X_{16} \le 503) = P\!\left(\frac{500-502}{0.625} \le Z \le \frac{503-502}{0.625}\right) = P(-3.2 \le Z \le 1.6).

Using standard normal tables: Φ(1.6)≈0.9452\Phi(1.6) \approx 0.9452, Φ(−3.2)=1−Φ(3.2)≈1−0.99931=0.00069\Phi(-3.2) = 1 - \Phi(3.2) \approx 1 - 0.99931 = 0.00069. Hence

P(500≤Xˉ16≤503)≈0.9452−0.00069=0.9445.P(500 \le \bar X_{16} \le 503) \approx 0.9452 - 0.00069 = 0.9445.

For the second part, we want cc with P(Xˉ16>c)=0.01P(\bar X_{16} > c) = 0.01, i.e. P(Z>(c−502)/0.625)=0.01P(Z > (c-502)/0.625) = 0.01. The upper 1% point of the standard normal is z0.01≈2.3263z_{0.01} \approx 2.3263. So

c−5020.625=2.3263⟹c=502+0.625(2.3263)≈502+1.4539=503.45 mL.\frac{c - 502}{0.625} = 2.3263 \quad\Longrightarrow\quad c = 502 + 0.625(2.3263) \approx 502 + 1.4539 = 503.45 \text{ mL}.

Note how the answer hinges on the exact result of part (b): because the population itself is normal, there is no approximation error at all, unlike Example 1 — a deliberate contrast between the exact-normal case (b) and the finite-sample non-normal case where only the CLT (c) or exact convolution is available.

CounterexampleDropping independence breaks the variance formula and the CLT

The theorem requires X1,…,XnX_1,\dots,X_n to be independent (not just identically distributed). Drop independence and both parts (a)'s variance formula and part (c)'s limit can fail. Concretely, let X1∼N(0,1)X_1 \sim N(0,1) and set X2=X3=⋯=Xn=X1X_2 = X_3 = \cdots = X_n = X_1 (perfectly, deterministically correlated — an extreme but legitimate example of identically distributed, dependent random variables with the same marginal law N(0,1)N(0,1) for each XiX_i). Then

Xˉn=1n∑i=1nXi=1n(nX1)=X1.\bar X_n = \frac{1}{n}\sum_{i=1}^n X_i = \frac{1}{n}(nX_1) = X_1.

So Xˉn∼N(0,1)\bar X_n \sim N(0,1) for every nn — its variance is exactly 11, not σ2/n=1/n\sigma^2/n = 1/n as claimed by part (a). The variance does not shrink at all as n→∞n \to \infty, and consequently the standardized statistic

Zn=Xˉn−01/n=n X1Z_n = \frac{\bar X_n - 0}{1/\sqrt n} = \sqrt n\, X_1

diverges in distribution (its variance is n→∞n \to \infty) rather than converging to N(0,1)N(0,1). Both the exact formula in (a) and the asymptotic normality in (c) fail completely once independence is removed, even though each XiX_i individually still has the "correct" marginal distribution N(0,1)N(0,1). This shows independence is not a technical nicety: it is exactly the ingredient that makes variances add (via vanishing covariances) and that makes the characteristic function of the sum factor into a product, which is the engine of the entire proof.

Scope, edge cases & common traps

SituationWhat holdsWhat can fail
XiX_i i.i.d. normal, any nnXˉn∼N(μ,σ2/n)\bar X_n \sim N(\mu,\sigma^2/n) exactlyNothing — exact for all n≥1n \ge 1
XiX_i i.i.d., finite σ2\sigma^2, nn largeXˉn\bar X_n approximately N(μ,σ2/n)N(\mu,\sigma^2/n) (CLT)Approximation quality depends on skewness/kurtosis of FF and on nn; can be poor for small nn or heavy tails
XiX_i i.i.d. but Var(Xi)=∞\mathrm{Var}(X_i) = \infty (e.g. Cauchy)E[Xˉn]E[\bar X_n], Var(Xˉn)\mathrm{Var}(\bar X_n) may not exist; CLT as stated does not applySampling distribution of Xˉn\bar X_n can equal the population distribution unchanged (Cauchy is stable) — no concentration as nn grows
XiX_i identically distributed but dependentVariance formula σ2/n\sigma^2/n and CLT limit generally failSee counterexample above
Statistic is not the mean (e.g. sample max, sample median)Has its own sampling distribution, generally not asymptotically normal at rate 1/n1/\sqrt n (e.g. sample maximum of Uniform(0,θ)(0,\theta) scales like 1/n1/n, not 1/n1/\sqrt n)Applying the CLT verbatim to non-mean statistics is a common error

The mathematician behind it

Origins · 1 minPierre-Simon Laplace and how this was discovered
PL
Pierre-Simon Laplace
1749–1827 · Paris, École Militaire and the Bureau des Longitudes

Laplace's 1810 memoir establishing an early general form of the Central Limit Theorem grew out of his work combining astronomical observations, where he needed to understand how errors from repeated measurements would aggregate and average out. He showed that sums of many small, independent errors tend toward a Gaussian shape regardless of the error's own distribution — the conceptual seed of what would become, a century later, the fully rigorous statement proved via characteristic functions by Lyapunov, Lindeberg, and Lévy.

History
- 1733 — Abraham de Moivre proves a special case (normal approximation to the binomial), the first CLT-type result.
- 1810 — Pierre-Simon Laplace publishes a general central limit result for sums of independent errors, motivating combination of astronomical observations.
- 1900 — Aleksandr Lyapunov gives the first fully rigorous proof of the CLT under general conditions using characteristic functions.
- 1920s — Jarl Waldemar Lindeberg and Paul Lévy refine the conditions (Lindeberg condition) and develop the continuity theorem for characteristic functions used above.
- 1908 — William Sealy Gosset ("Student") studies the exact sampling distribution of a mean-based statistic for small normal samples, motivating the tt-distribution.
What exactly is the difference between a "statistic" and a "parameter"?

A parameter (like the population mean μ\mu or variance σ2\sigma^2) is a fixed, typically unknown, number describing the population distribution itself; it is not random. A statistic is a function of the observed sample data alone — it must be computable from the data without knowledge of any unknown parameter, and because the sample is random, the statistic is a random variable with its own distribution, the sampling distribution. The entire enterprise of inference is using the known, computable sampling distribution of a statistic to make calibrated statements about the unknown, fixed parameter.

Does the Central Limit Theorem say that the population distribution becomes normal as n grows?

No — this is a very common misreading. The population distribution FF never changes; it is fixed throughout. What changes with nn is the sampling distribution of the sample mean Xˉn\bar X_n, which becomes approximately normal as n→∞n \to \infty, even though each individual observation XiX_i still comes from the original, possibly very non-normal, population FF. The CLT is a statement about an aggregated statistic, not about the underlying data-generating distribution.

Why does the exact normal result (part b) require the population itself to be normal, while the CLT (part c) does not?

Part (b)'s proof uses the specific algebraic form of the normal MGF, exp⁡(μt+σ2t2/2)\exp(\mu t + \sigma^2 t^2/2), whose nn-th power is again exactly a normal MGF — a special closure property unique to (a few distributions including) the normal family. Part (c) instead only uses the second-order Taylor expansion of the characteristic function near t=0t=0, which exists for any distribution with finite variance; the higher-order terms of φ\varphi (which do depend on the population's exact shape) vanish in the limit n→∞n\to\infty because they are evaluated at t/n→0t/\sqrt n \to 0. This is exactly why (c) is asymptotic and shape-independent, while (b) is exact but shape-specific.

If n is only moderately large, how do I know whether the normal approximation from the CLT is trustworthy?

There is no universal cutoff, but rough rules of thumb exist: for populations that are roughly symmetric and light-tailed, n≳20n \gtrsim 20–3030 often gives a good approximation; for strongly skewed populations (like the exponential in Example 1) or heavy-tailed ones, much larger nn may be needed, and for infinite-variance populations (e.g. Cauchy) the CLT as stated does not apply at all regardless of nn. When an exact sampling distribution is available (as with normal populations, or sums of exponentials giving a Gamma), it should always be preferred over the normal approximation for small or moderate nn.

Is the sampling distribution the same thing as the "sample distribution" or empirical distribution of the data?

No, and conflating the two is a frequent source of confusion. The empirical distribution is the discrete distribution putting mass 1/n1/n on each of the nn observed data points in one particular sample; it is a description of the data you actually have. The sampling distribution instead describes how a statistic (like the mean of that data) would vary if you repeated the entire sampling procedure indefinitely; it is a theoretical object describing variability across hypothetical repeated samples, not the variability within one sample.

Why is variance divided by n, rather than staying constant, in the sampling distribution of the mean?

Because averaging cancels random fluctuations: independent errors partially offset each other when summed, and dividing by nn shrinks the resulting spread further. The precise mechanism, shown in the proof of part (a), is that Var(∑Xi)=nσ2\mathrm{Var}(\sum X_i) = n\sigma^2 grows only linearly in nn (since cross terms vanish by independence), while dividing the sum by nn to form the mean scales the variance down by n2n^2, leaving a net factor of σ2/n\sigma^2/n. This is precisely why "more data" translates mathematically into a narrower, more concentrated sampling distribution.

How does this concept connect to the sample variance and to confidence intervals?

The sample mean is only one example of a statistic with a well-understood sampling distribution; the same conceptual machinery — treat the statistic as a random variable, find its distribution across repeated sampling — applies to the sample variance Sn2S_n^2, whose exact sampling distribution under normality involves the chi-squared distribution (see The t, Chi-Squared, and F Distributions). Confidence intervals are constructed by inverting a known sampling distribution: if we know that a standardized statistic like ZnZ_n falls in a certain range with 95% probability across repeated sampling, we can algebraically solve for a range of plausible parameter values consistent with the one sample we actually observed — the core idea developed in Confidence Intervals: Concept and Construction.

Before this

Where this leads