Central Limit Theorem

The Central Limit Theorem is the deepest structural result in elementary probability and the single reason the Gaussian curve is the default shape of physics. Whenever a measured quantity is the accumulated effect of many independent microscopic contributions — the thermal voltage across a resistor, the net displacement of a Brownian particle, the residual error left after averaging repeated measurements — the result is approximately normal, and the CLT is the precise statement of why — it turns the messy particulars of any noise source into one two-parameter bell. The setting is a sequence of independent, identically distributed random variables $X_1,X_2,\dots$ with a common mean $\mu=\mathbb{E}[X_i]$ and finite variance $\sigma^2=\mathrm{Var}(X_i)$ . Their sum $S_n=\sum_{i=1}^n X_i$ has mean $n\mu$ and variance $n\sigma^2$ , so it both drifts and spreads as $n$ grows; to see a stable shape one recenters by $n\mu$ and rescales by the standard deviation $\sigma\sqrt{n}$ . The recentered, rescaled variable stops depending on $n$ and freezes into the standard normal — no matter what the individual $X_i$ looked like. The companion Law of Large Numbers says the sample mean $S_n/n$ collapses onto the constant $\mu$ ; the CLT is the magnified, second-order view of that collapse, resolving the $O(1/\sqrt{n})$ fluctuations around $\mu$ and giving them their universal Gaussian shape. The tool that makes this transparent is the characteristic function $\varphi_X(t)=\mathbb{E}[e^{itX}]$ , which converts the convolution of summed distributions into ordinary multiplication; Lévy's continuity theorem then carries convergence of the transforms back to convergence of distributions.

Key takeaways

The Central Limit Theorem says the suitably centered and rescaled sum of many independent random variables tends to a Gaussian, whatever the individual distributions, provided they have finite variance.
For i.i.d. $X_i$ with mean $\mu$ and variance $\sigma^2$ , the standardized sum $Z_n=\frac{1}{\sigma\sqrt{n}}\sum_{i=1}^n(X_i-\mu)$ converges in distribution to the standard normal $\mathcal{N}(0,1)$ .
The scaling is forced: $\mathrm{Var}(S_n)=n\sigma^2$ , so only division by $\sigma\sqrt{n}$ gives a fixed unit variance — divide by $n$ instead and you get the degenerate Law-of-Large-Numbers limit.
The cleanest proof lives in characteristic-function space: independence turns sums into products, and the Taylor expansion of $\log\varphi$ keeps only the quadratic (variance) term after rescaling.
Convergence is of the CDF, not of the random variable: the fluctuations of size $\sigma\sqrt{n}$ never shrink, only their standardized shape stabilizes; finite variance is essential, or heavy tails send you to non-Gaussian stable laws (the rate is $O(1/\sqrt{n})$ , slowest for skewed terms).
Key idea: adding many independent finite-variance random variables and rescaling by 1/√n erases every feature of the individual distributions except their mean and variance, leaving a universal Gaussian — because in characteristic-function space independence turns sums into products and only the quadratic term of log φ survives the limit.

The core insight

Adding many independent finite-variance random variables and rescaling the centered sum by $1/\sqrt{n}$ erases every feature of the individual distributions except their mean and variance, leaving a universal Gaussian — most transparently in characteristic-function space, where independence turns sums into products and only the quadratic (variance) term of $\log\varphi$ survives the limit.

Where this lives today

The theorem is the quiet workhorse behind almost every quantitative error bar. Monte Carlo integration — the standard method for high-dimensional integrals in lattice QCD, statistical mechanics, and computational finance — owes its whole error analysis to the CLT: an average of $N$ samples has a Gaussian statistical error falling as $\sigma/\sqrt{N}$ , which is why results carry a one-sigma band and why halving the error costs four times the compute. In gravitational-wave detection, the matched-filtering pipelines at LIGO and Virgo treat the instrument noise as a sum of vast numbers of independent disturbances, approximately Gaussian by the CLT, so a candidate is graded by how many standard deviations it stands above that background — the "five-sigma" bar shared with particle physics. The same logic underwrites the margins quoted in opinion polls, the bootstrap in modern statistics, and the diffusion-limited pricing of options. Where the assumptions break — heavy-tailed returns, long-range correlations near a critical point, bursty network traffic — the failure is just as informative, pushing the field toward stable laws, extreme-value statistics, and the anomalous non-Gaussian scaling that signals genuinely collective behaviour.

The physical picture

Think of each $X_i$ as a small random nudge and their sum as a one-dimensional random walk. After $n$ steps the walk has drifted a distance $n\mu$ and spread by a typical amount $\sigma\sqrt{n}$ — the diffusive $\sqrt{n}$ , not the ballistic $n$ . Zoom so this spread becomes unit size, measuring position in units of $\sigma\sqrt{n}$ about the drift, and the histogram always settles into the same bell — whether the individual step was a coin flip, a die roll, or a lopsided exponential. The first figure shows this "forgetting" happen in front of you: the flat single-die distribution convolves toward a Gaussian in just a handful of steps.

Figure

Sum of n fair dice: the flat one-die distribution (amber) convolves to a triangle at n = 2 (green) and to a smooth bell at n = 5 (blue). The mean grows as 3.5n and the variance as 2.917n, so the spread widens only as √n while the shape forgets the die and becomes Gaussian.

In characteristic-function language the mechanism is purely algebraic. Independence turns the characteristic function of the sum into a product, $\varphi_{S_n}=\prod_i\varphi_{X_i}$ , so $\log\varphi$ becomes a sum whose first nonconstant term, for a centered variable, is the quadratic $-\tfrac12\sigma^2 t^2$ ; higher terms carry skewness, kurtosis, and every other detail. The $1/\sqrt{n}$ rescaling is engineered so the $n$ -fold sum exactly balances that quadratic — it survives, while cubic and higher terms are divided down to zero. Only the two lowest cumulants, mean and variance, live through the limit — the "forgetting" that makes the Gaussian a stable attractor, the probabilistic image of a renormalization-group fixed point. The second figure is the payoff: a discrete binomial read straight off the universal bell.

Figure

Universal limit in action: the standardized binomial S₁₀₀ with p = 0.5 has mean np = 50 and standard deviation √npq = 5, so it is read off the standard normal N(0,1). With the continuity correction z = (60.5 − 50)/5 = 2.1, the shaded area gives P(S ≤ 60) ≈ Φ(2.1) = 0.9821, against the exact binomial 0.9824.

The core equations

The theorem itself is a single line of convergence: standardize the sum by subtracting its mean and dividing by its standard deviation $\sigma\sqrt{n}$ , and the result tends in distribution to the standard normal. Everything else in this article either derives this line, quantifies how fast it holds, or relaxes its hypotheses.

Z_n=\frac{1}{\sigma\sqrt{n}}\sum_{i=1}^{n}\left(X_i-\mu\right)\xrightarrow{\,d\,}\mathcal{N}(0,1) \qquad (1)

where: $X_i$ — i.i.d. random variables with mean $\mu$ and variance $\sigma^2$ ; $\mu$ — common mean $\mathbb{E}[X_i]$ ; $\sigma^2$ — common variance $\mathrm{Var}(X_i)$ , assumed finite and nonzero; $n$ — number of summands; $\xrightarrow{d}$ — convergence in distribution; $\mathcal{N}(0,1)$ — standard normal, density $\tfrac{1}{\sqrt{2\pi}}e^{-z^2/2}$ . Reading: the standardized sum's shape stops depending on $n$ or on the details of the $X_i$ . "Convergence in distribution" means precisely that the cumulative distribution function of $Z_n$ approaches that of the standard normal at every point. This is the honest, checkable form of the statement, and the one used to read off probabilities in practice.

\lim_{n\to\infty}P\!\left(Z_n\le z\right)=\Phi(z)=\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{z}e^{-u^2/2}\,\mathrm{d}u \qquad (2)

where: $\Phi(z)$ — standard normal cumulative distribution function; $z$ — any real argument (every point is a continuity point, since $\Phi$ is continuous). To get a probability, standardize the boundary and read $\Phi$ . The engine of the proof is the characteristic function, the Fourier transform of the distribution. Its defining property is that it linearizes sums: for independent variables the characteristic function of a sum is the product of the individual characteristic functions, converting a convolution into a multiplication.

\varphi_X(t)=\mathbb{E}\!\left[e^{itX}\right],\qquad \varphi_{X+Y}(t)=\varphi_X(t)\,\varphi_Y(t)\ \ \text{(independent)} \qquad (3)

where: $\varphi_X(t)$ — characteristic function of $X$ ; $t$ — real Fourier-conjugate variable; $X,Y$ — independent random variables. Independence makes the transform multiplicative, and multiplicativity makes the $n$ -fold sum tractable. Applying this to the standardized sum turns the whole problem into a single factor raised to the $n$ -th power. Expanding that factor to second order and taking the limit yields the characteristic function of the Gaussian — this equation is the compressed statement of the proof.

\varphi_{Z_n}(t)=\left[\varphi_{Y}\!\left(\frac{t}{\sigma\sqrt{n}}\right)\right]^{n}\xrightarrow{\,n\to\infty\,}e^{-t^2/2} \qquad (4)

where: $Y=X-\mu$ — centered variable; $\varphi_Y$ — its characteristic function; $e^{-t^2/2}$ — the characteristic function of $\mathcal{N}(0,1)$ , the signature that only the quadratic term of $\log\varphi_Y$ survives the rescaling. For everyday estimation one rarely standardizes by hand; instead one uses the practical corollary that a large sum is approximately Gaussian with the sum's own mean and variance. This is the form invoked when quoting error bars.

S_n\approx\mathcal{N}\!\left(n\mu,\;n\sigma^2\right),\qquad \bar X=\frac{S_n}{n}\approx\mathcal{N}\!\left(\mu,\;\frac{\sigma^2}{n}\right) \qquad (5)

where: $S_n$ — unscaled sum; $\bar X$ — sample mean; $n\mu$ and $n\sigma^2$ — mean and variance of the sum (additivity of expectation and, for independent terms, of variance); $\sigma/\sqrt{n}$ — the standard error of the mean, shrinking as $1/\sqrt{n}$ . The theorem asserts convergence but not its speed; the Berry–Esseen theorem supplies the rate, bounding the worst-case gap between the true CDF and the Gaussian by a term that decays as $1/\sqrt{n}$ and grows with the skewness of the summands.

\sup_{x}\left|F_n(x)-\Phi(x)\right|\le \frac{C\,\rho}{\sigma^3\sqrt{n}},\qquad \rho=\mathbb{E}\,|X-\mu|^3 \qquad (6)

where: $F_n$ — CDF of the standardized sum $Z_n$ ; $\rho$ — absolute third central moment, assumed finite; $C$ — a universal constant (best known bound $C<0.4748$ ). Reading: convergence is $O(1/\sqrt{n})$ , slower the more skewed the summands. Finally, the identical-distribution hypothesis can be dropped: the Lindeberg–Lyapunov form asks only that no single term dominate the total variance — the technical content of "many small independent contributions."

\frac{1}{s_n}\sum_{i=1}^{n}(X_i-\mu_i)\xrightarrow{\,d\,}\mathcal{N}(0,1),\qquad s_n^2=\sum_{i=1}^n\sigma_i^2 \qquad (7)

where: $\mu_i,\sigma_i^2$ — individual (no longer identical) means and variances; $s_n^2$ — total variance. Reading: asymptotically Gaussian provided $\max_i\sigma_i^2/s_n^2\to0$ , so each term is negligible.

Derivation AProof of the classical theorem by characteristic functions

The goal is to prove equation (1) in Fourier space, where the awkward convolution of summed distributions becomes plain multiplication. The plan is three moves: standardize so the variance is pinned at one, use independence to collapse the characteristic function of the sum into a single factor raised to the n-th power, and Taylor-expand that factor to see that only the variance term survives — after which Lévy's continuity theorem carries us back from transforms to distributions. Let $X_1,\dots,X_n$ be i.i.d. with mean $\mu$ and finite variance $\sigma^2$ , set $Y_i=X_i-\mu$ so $\mathbb{E}[Y_i]=0$ and $\mathrm{Var}(Y_i)=\sigma^2$ , and define $Z_n=\frac{1}{\sigma\sqrt{n}}\sum_i Y_i$ .

Step 1. Write out the characteristic function of $Z_n$ and factorize it: $\varphi_{Z_n}(t)=\mathbb{E}\big[\exp(\frac{it}{\sigma\sqrt{n}}\sum_i Y_i)\big]=\prod_{i=1}^n\varphi_Y(\frac{t}{\sigma\sqrt{n}})=\big[\varphi_Y(\frac{t}{\sigma\sqrt{n}})\big]^n$ . Why: independence makes the expectation of a product equal the product of expectations, and identical distribution makes every factor the same, so an n-term sum becomes a single factor raised to the n-th power.

Step 2. Expand the single factor for small argument. Because $Y$ has two finite moments with $\mathbb{E}[Y]=0$ and $\mathbb{E}[Y^2]=\sigma^2$ , Taylor's theorem for characteristic functions gives $\varphi_Y(s)=1-\tfrac12\sigma^2 s^2+o(s^2)$ as $s\to0$ . Why: the linear term dies because $Y$ is centered, and two finite moments are exactly enough to control the remainder — no higher moments are needed for the classical theorem.

Step 3. Substitute the rescaled argument $s=t/(\sigma\sqrt{n})$ : the variance cancels precisely, $\varphi_Y(\frac{t}{\sigma\sqrt{n}})=1-\frac{\sigma^2}{2}\cdot\frac{t^2}{\sigma^2 n}+o(\frac1n)=1-\frac{t^2}{2n}+o(\frac1n)$ . Why: this exact cancellation of $\sigma^2$ against the $1/\sigma^2$ from standardization is the reason the scaling must be $\sigma\sqrt{n}$ and nothing else — it leaves a clean $t^2/2n$ blind to the microscopic variance.

Step 4. Raise to the n-th power and take the limit: $\big[1-\frac{t^2}{2n}+o(\frac1n)\big]^n\to e^{-t^2/2}$ . Why: this is the compound-interest limit $(1+a_n/n)^n\to e^a$ ; the higher cumulants sit in the $o(1/n)$ and are scaled away as $n^{-1/2}$ , which is exactly where the skewness and kurtosis of the original distribution vanish.

Step 5. Identify $e^{-t^2/2}$ as the characteristic function of $\mathcal{N}(0,1)$ and invoke Lévy's continuity theorem: pointwise convergence of $\varphi_{Z_n}(t)$ to a function continuous at $t=0$ forces $Z_n\xrightarrow{d}\mathcal{N}(0,1)$ . Why: characteristic functions determine distributions uniquely, and Lévy's theorem is the precise license to pass from convergence of the transforms back to convergence of the CDFs. *Note: the limit step rests on the standard result $(1-t^2/2n)^n\to e^{-t^2/2}$ , and $e^{-t^2/2}$ is indeed the canonical Fourier transform $\int e^{itx}\tfrac{1}{\sqrt{2\pi}}e^{-x^2/2}\,dx$ of the normal density.*

Go deeper · Derivation B · 1 minMean and variance of a sum and the scaling that demands root-n

Derivation BMean and variance of a sum and the scaling that demands root-n

The goal is to show, before any distributional limit, that the standardization in equation (1) is forced by the moments of the sum: the standard deviation of $S_n$ grows like $\sigma\sqrt{n}$ , so only division by $\sigma\sqrt{n}$ gives a non-degenerate limit. We compute the first two moments of the unscaled sum $S_n=\sum_{i=1}^n X_i$ for independent $X_i$ with common mean $\mu$ and variance $\sigma^2$ , then read off the unique scaling that holds the variance at one.

Step 1. The mean follows from linearity alone: $\mathbb{E}[S_n]=\sum_{i=1}^n\mathbb{E}[X_i]=n\mu$ . Why: linearity of expectation needs no independence, so the drift of the walk is simply proportional to the number of steps.

Step 2. The variance uses independence: $\mathrm{Var}(S_n)=\sum_i\mathrm{Var}(X_i)+\sum_{i\ne j}\mathrm{Cov}(X_i,X_j)=n\sigma^2$ , since every covariance vanishes. Why: independence kills the cross terms, so variances add — this additivity is exactly the property that fails for correlated summands and can break the theorem.

Step 3. Hence the standard deviation is $\mathrm{SD}(S_n)=\sigma\sqrt{n}$ : the typical fluctuation grows only as $\sqrt{n}$ while the mean grows as $n$ . Why: this is the diffusive law of the random walk — spread lagging drift — and it is the physical fingerprint of the $\sqrt{n}$ in the theorem.

Step 4. Test the standardization: $\mathrm{Var}\big(\frac{S_n-n\mu}{\sigma\sqrt{n}}\big)=\frac{n\sigma^2}{\sigma^2 n}=1$ for every $n$ . Why: only division by $\sigma\sqrt{n}$ freezes the variance at one; dividing by $n$ gives $\sigma^2/n\to0$ (the degenerate Law-of-Large-Numbers limit), and dividing by less gives $n\sigma^2\to\infty$ — the CLT is the unique magnification that holds the picture still. *Note: this rests on standard additivity of variance for independent variables; the $\sqrt{n}$ spreading is the same diffusion law $\langle x^2\rangle\propto t$ that governs a random walk.*

Go deeper · Derivation C · 1 minde Moivre and Laplace and the Gaussian limit of the binomial

Derivation Cde Moivre and Laplace and the Gaussian limit of the binomial

The goal is to specialize the general theorem to the oldest concrete case — repeated coin flips — and recover the de Moivre–Laplace result that a binomial count is asymptotically Gaussian, together with the continuity correction that makes it accurate at moderate $n$ . Let $X_i$ be i.i.d. Bernoulli $(p)$ , taking value $1$ with probability $p$ and $0$ with probability $q=1-p$ , so $\mu=p$ and $\sigma^2=p(1-p)=pq$ ; then the count $S_n=\sum_i X_i$ is exactly Binomial $(n,p)$ .

Step 1. Read off the moments from the Bernoulli values: $\mathbb{E}[S_n]=np$ and $\mathrm{Var}(S_n)=npq$ . Why: this is just the sum-scaling result of the previous derivation with $\mu=p$ and $\sigma^2=pq$ substituted in.

Step 2. Apply the classical theorem directly, since Bernoulli variables have all moments finite: $Z_n=\frac{S_n-np}{\sqrt{npq}}\xrightarrow{d}\mathcal{N}(0,1)$ . Why: nothing in the hypotheses fails, so the binomial count, once centered at $np$ and scaled by $\sqrt{npq}$ , is asymptotically standard normal.

Step 3. Upgrade from the CDF to the individual probabilities using the local limit theorem: $P(S_n=k)\approx\frac{1}{\sqrt{2\pi npq}}\exp\!\big[-\frac{(k-np)^2}{2npq}\big]$ for $k$ within $O(\sqrt{npq})$ of $np$ . Why: the discrete probability mass at each $k$ is well approximated by the Gaussian density times the unit spacing between integers, which is the density form of the theorem.

Step 4. Correct for discreteness when summing a range: $P(a\le S_n\le b)\approx\Phi\!\big(\frac{b+\frac12-np}{\sqrt{npq}}\big)-\Phi\!\big(\frac{a-\frac12-np}{\sqrt{npq}}\big)$ . Why: replacing a bar of width one by an area under a smooth curve requires extending each integer boundary by half a unit; this continuity correction markedly improves accuracy at moderate $n$ , as the worked example confirms to four decimals. *Note: this is the original 1733/1812 de Moivre–Laplace theorem, reliable when $np>5$ and $nq>5$ ; when instead $p\to0$ with $np$ fixed the binomial degenerates to the Poisson limit rather than the Gaussian.*

Limiting cases

The CLT is the central member of a family of limit laws; changing one hypothesis moves you to a neighbour, each with a different attractor:

Case	Result	Meaning
Finite variance, $n\to\infty$	$Z_n=\frac{S_n-n\mu}{\sigma\sqrt{n}}\xrightarrow{d}\mathcal{N}(0,1)$	The standard regime: the standardized sum is asymptotically Gaussian, whatever the shape of the $X_i$ .
Infinite variance (heavy tails)	$\frac{S_n-b_n}{a_n}\xrightarrow{d}\text{stable law}$	With $\sigma^2=\infty$ (e.g. Cauchy) the CLT fails; the limit is a non-Gaussian stable law. The Cauchy is its own attractor — the mean of $n$ Cauchy variables is again Cauchy.
Rare events, $np=\lambda$ fixed	$\mathrm{Bin}(n,p)\to\mathrm{Poisson}(\lambda)$	When $p\to0$ as $n\to\infty$ with $np$ fixed the binomial tends to Poisson, not Gaussian — the per-term variance shrinks too fast for the CLT to bite.
Divide by $n$ not $\sqrt{n}$	$\frac{S_n}{n}\xrightarrow{P}\mu$	The Law of Large Numbers: the sample mean collapses onto the constant $\mu$ . The CLT is the magnified view of the fluctuations around this degenerate limit.

Common trapThe sum itself becomes Gaussian — no rescaling needed

This is the most common misreading. The CLT is about convergence in distribution of the standardized sum, not the raw sum. The unscaled $S_n$ never settles onto a fixed Gaussian: its mean $n\mu$ drifts off and its variance $n\sigma^2$ grows without bound, so it has no limiting distribution at all. Only after subtracting $n\mu$ and dividing by $\sigma\sqrt{n}$ does a limit appear — and even then it is the CDF that converges, not the random variable, which keeps fluctuating on the ever-growing scale $\sigma\sqrt{n}$ . Two further traps ride along. Finite variance is essential: heavy-tailed variables such as the Cauchy have no finite $\sigma^2$ and obey stable-law limits instead. And "large $n$ " does not buy uniform accuracy — the center converges far faster than the tails, so an approximation trusted at $z=1$ can be badly wrong at $z=5$ , where rare events live.

Worked examples

1 · Normal approximation to a binomial (confidence builder)

A fair coin is flipped $n=100$ times. Estimate $P(S_{100}\le 60)$ , the probability of at most 60 heads, using the CLT with a continuity correction, and compare to the exact binomial.

Step 1. Identify the moments. With $p=0.5$ , the mean is $np=100\times0.5=50$ and the variance is $npq=100\times0.5\times0.5=25$ , so the standard deviation is $\sqrt{npq}=\mathbf{5}$ . The count $S_{100}$ is therefore approximately $\mathcal{N}(50,25)$ .

Step 2. Standardize the boundary with the continuity correction. Because $S_{100}$ is integer-valued, " $\le 60$ " reaches to the midpoint $60.5$ : $z=\dfrac{60.5-50}{5}=\dfrac{10.5}{5}=\mathbf{2.1}$ . (Without the correction one would use $z=(60-50)/5=2.0$ .)

Step 3. Read off the Gaussian probability: $P(S_{100}\le 60)\approx\Phi(2.1)=\mathbf{0.9821}$ . The exact binomial sum is $0.9824$ , so the corrected approximation is right to three decimal places; the uncorrected $\Phi(2.0)=0.9772$ is noticeably worse. The continuity-corrected CLT gives P ≈ 0.9821 against the exact 0.9824 — a 0.0003 error.

2 · Adding twelve uniforms to make a standard normal

A classic random-number recipe: draw twelve independent numbers $U_1,\dots,U_{12}$ uniformly on $[0,1]$ and form $Z=\sum_{i=1}^{12}U_i-6$ . Show that $Z$ is an approximate standard normal, and estimate $P(Z\le 1)$ .

Step 1. Moments of one uniform. For $U\sim\mathrm{Uniform}(0,1)$ , $\mathbb{E}[U]=\tfrac12$ and $\mathrm{Var}(U)=\tfrac{1}{12}$ . These are the building blocks the CLT will sum.

Step 2. Moments of the sum. By additivity, $\mathbb{E}\big[\sum_{i=1}^{12}U_i\big]=12\times\tfrac12=6$ and $\mathrm{Var}\big[\sum_{i=1}^{12}U_i\big]=12\times\tfrac{1}{12}=\mathbf{1}$ . So $Z=\sum U_i-6$ has $\mathbb{E}[Z]=0$ and $\mathrm{Var}(Z)=1$ — already standardized, which is exactly why twelve was chosen: the divisor $\sigma\sqrt{n}=\sqrt{1}=1$ is invisible.

Step 3. Invoke the CLT. Twelve i.i.d. finite-variance terms give a good Gaussian fit near the center, so $Z\approx\mathcal{N}(0,1)$ and $P(Z\le 1)\approx\Phi(1)=\mathbf{0.8413}$ . Summing twelve uniforms and subtracting six yields a ready-made N(0,1); P(Z ≤ 1) ≈ 0.8413. The construction truncates beyond $\pm6$ (the range of $Z$ ), a reminder that the tails fray first. The same $\sigma/\sqrt{n}$ scaling shrinks the standard error of an averaged measurement: 25 readings of noise $\sigma$ give a mean of spread $\sigma/5$ and a 95% interval $\pm1.96\,\sigma/\sqrt{n}$ .

Physics of the portrait

The Central Limit Theorem is less one discovery than a two-century relay: a lucky special case for coin-tossing, a sweeping but non-rigorous generalization, and finally the modern proof and its exact conditions.

Origins · 1 mincentral limit theorem and how this was discovered

central limit theorem

1733 to 1935, de Moivre to Lévy · the universal Gaussian limit of independent sums

The theorem grew by accretion. Abraham de Moivre, working on games of chance, found in 1733 that the binomial for a fair coin is approximated by what we now call the normal curve — the first appearance of $e^{-x^2/2}$ in probability, though he had no notion of it as a universal law. Pierre-Simon Laplace absorbed and vastly extended the idea in his 1812 Théorie analytique des probabilités, applying the Gaussian approximation to sums of many small errors and effectively stating the theorem in the generality physicists still use — but his arguments, like Gauss's contemporaneous work on the error law, were formal rather than rigorous. The proof a modern analyst would accept waited for Aleksandr Lyapunov, who in 1901 introduced the characteristic-function method and a moment condition guaranteeing the limit — the template for the argument above. Jarl Waldemar Lindeberg sharpened the hypothesis in 1922 to the essentially minimal "no single term dominates" condition, and Paul Lévy with William Feller closed the question in the 1930s by proving that condition necessary as well as sufficient. It was George Pólya who, in 1920, gave the result its enduring name — zentraler Grenzwertsatz — with "central" meaning central to probability itself. That finite variance is exactly the price of admission to the Gaussian is the joint legacy of them all.

History

1733

— de Moivre finds the normal curve as the limit of the binomial for a fair coin

1812

— Laplace generalizes the result in Théorie analytique des probabilités

1901

— Lyapunov gives the first rigorous proof by characteristic functions

1920

— Pólya names it the central limit theorem

1922

— Lindeberg proves it under his minimal sufficiency condition

Going further

Texts. Stone & Goldbart, Mathematics for Physics, chapter on probability and statistics · Arfken, Weber & Harris, Mathematical Methods for Physicists, 7th ed., probability chapter · Riley, Hobson & Bence, Mathematical Methods for Physics and Engineering, 3rd ed., Ch. 30 · Feller, An Introduction to Probability Theory and Its Applications, Vols. 1 and 2 · Grimmett & Stirzaker, Probability and Random Processes, 3rd ed., Ch. 5. Key results used throughout: the Gaussian characteristic function $e^{-t^2/2}$ is the Fourier transform of $\tfrac{1}{\sqrt{2\pi}}e^{-x^2/2}$ ; variance additivity gives $\mathrm{Var}(S_n)=n\sigma^2$ for independent terms; the Berry–Esseen best-known constant is $C<0.4748$ . The numerical values are $\Phi(2.1)=0.9821$ , exact $\mathrm{Bin}(100,0.5)$ tail $=0.9824$ , $\Phi(1.0)=0.8413$ , and $\Phi(1.96)=0.975$ .

Prerequisites. Random Variables and Probability Distributions · Expectation, Variance and Moments · Moment-Generating and Characteristic Functions · Gaussian (Normal) Distribution · Laws of Large Numbers and Convergence of Random Variables. Leads to. Multivariate Gaussian Distribution · Poisson Distribution · Binomial and Bernoulli Distributions.

Ask the tutor: Ask me to run the twelve-uniform sum through the characteristic-function proof step by step, or to show how the continuity correction sharpens a binomial estimate you have in mind.

Does the central limit theorem require the summands to be identically distributed?

No — that is only the simplest version. The classical Lindeberg–Lévy form assumes i.i.d. variables, but the Lindeberg and Lyapunov generalizations drop the "identical" requirement. All they demand is that the terms be independent, have finite variances, and satisfy a smallness condition: no single term may contribute a finite fraction of the total variance, i.e. $\max_i\sigma_i^2/s_n^2\to0$ . This is what lets the theorem apply to real physical noise, which is almost never a sum of identical pieces — thermal, shot, and flicker contributions differ, yet their sum is Gaussian as long as none dominates.

Why divide by √n rather than by n?

Because the variance of the centered sum grows like $n$ , so its standard deviation grows like $\sqrt{n}$ ; dividing by exactly $\sigma\sqrt{n}$ holds the variance at one for every $n$ , the only way to a limit that is neither degenerate nor divergent. Divide by $n$ and you are looking at the sample mean, whose variance $\sigma^2/n$ shrinks to zero — the Law of Large Numbers, collapsing the distribution to a spike at $\mu$ . Divide by something smaller than $\sqrt{n}$ and the variance blows up, with no limiting shape. The $\sqrt{n}$ is the unique magnification that keeps the fluctuations in view.

What actually converges — the random variable or its distribution?

Its distribution, specifically its cumulative distribution function. The theorem says $P(Z_n\le z)\to\Phi(z)$ at every $z$ ; it says nothing about the random variable $Z_n$ approaching any fixed random variable pointwise. This is "convergence in distribution," the weakest and most robust mode of convergence. The distinction matters: the underlying sum keeps fluctuating on the growing scale $\sigma\sqrt{n}$ and never settles as a number, yet the standardized shape of those fluctuations freezes into the bell. Confusing the two leads people to expect the raw sum to "become Gaussian," which it does not.

When is n large enough for the normal approximation to be trustworthy?

There is no universal threshold — it depends on the shape of the summands and on which part of the distribution you care about. For nearly symmetric, light-tailed variables it is already good by $n\approx10$ – $20$ near the center; the twelve-uniform recipe in the examples is one instance. Skewed distributions converge more slowly, and the Berry–Esseen bound $C\rho/(\sigma^3\sqrt{n})$ makes this quantitative through the third absolute moment. The common binomial rule of thumb, $np>5$ and $nq>5$ , is exactly the statement that the count sits far enough from its floor of zero to look symmetric. Crucially the tails always lag the center, so rare-event probabilities need much larger $n$ than typical ones.

Does the theorem explain why the Gaussian is everywhere in physics?

Largely, yes. Any quantity that is the additive result of many small, independent causes inherits a Gaussian distribution by the CLT, whatever the law of the individual causes — which is why measurement errors, thermal noise, and the fluctuations of extensive thermodynamic variables are all bell-shaped. The universality is a genuine "forgetting": the limit keeps only the mean and variance, much as a renormalization-group flow forgets microscopic structure at a fixed point. The caveat is "additive and independent." Where contributions multiply you get log-normal distributions, and where they are strongly correlated or heavy-tailed — near critical points, in turbulence, in some financial data — the Gaussian gives way to power laws.

How is the central limit theorem related to the Law of Large Numbers?

They are two orders of the same expansion of the sample mean $\bar X=S_n/n$ . The Law of Large Numbers is the leading term: $\bar X\to\mu$ , telling you where the estimator lands but nothing about its error. The CLT is the next term: the deviation $\bar X-\mu$ , magnified by $\sqrt{n}$ , is asymptotically $\mathcal{N}(0,\sigma^2)$ , so $\bar X\approx\mathcal{N}(\mu,\sigma^2/n)$ . In one sentence, the LLN gives the location of an estimator and the CLT the size and Gaussian shape of its fluctuations — together the foundation of statistical inference.