The Normal Distribution
Why does the same bell-shaped curve appear in human heights, physicists' experimental errors, and communication-channel noise? This recurring pattern is the Normal (or Gaussian) distribution. Understanding why it arises so universally is one of the deepest facts in probability. The idea matters because the Normal distribution is the load-bearing wall of classical statistics. Confidence intervals, hypothesis tests, and regression theory all rest on it. It is also the limiting shape guaranteed by the Central Limit Theorem for sums of many independent small effects. The big picture is simple. A single family of curves, indexed only by a mean and a variance , captures an enormous range of natural and man-made randomness. Every member of this family comes from one canonical curve — the standard normal. It is obtained by shifting and rescaling. This is why the technique of standardization is so powerful. It converts any question about any Normal random variable into a question about a single, extensively tabulated function. This article builds the Gaussian density from its defining formula. It proves that this density is a genuine probability density. It computes the mean and variance rigorously. It also establishes the standardization theorem, which reduces all Normal probability calculations to the standard normal CDF . We build on Continuous Random Variables and PDFs. The tools developed here — moment computations, the change-of-variables technique — feed directly into Moment Generating Functions and the general theory of Functions of Random Variables.
- The Normal density is a genuine PDF for any , .
- Its mean is and its variance is ; these parameters are the distribution's two degrees of freedom.
- Standardization converts any Normal variable into a Normal variable.
- The standard normal CDF has no elementary closed form and must be tabulated or computed numerically.
- Linear combinations and sums of independent Normals are again Normal — a rare and special closure property.
- Key idea: every Gaussian is a shifted, rescaled copy of one universal curve, so all computations reduce to .
Where this lives today
The Normal distribution underlies essentially all of classical inferential statistics: confidence intervals for means, -tests, ANOVA, and linear regression all assume Normal (or asymptotically Normal, via the Central Limit Theorem) error structure. In physics, the Maxwell–Boltzmann velocity distribution of gas molecules has Normal marginal components, and Gaussian noise models are the default assumption in signal processing and communications engineering, where channel noise is modeled as additive white Gaussian noise (AWGN). In machine learning, Gaussian distributions appear as priors in Bayesian inference, as the assumed noise model in linear regression's least-squares justification, and as the building block of Gaussian mixture models and Gaussian processes. In finance, the Black–Scholes option pricing model assumes log-normal (hence Normal-derived) asset price movements. In cryptography, Gaussian sampling is used in lattice-based post-quantum schemes to add carefully calibrated noise (as in the Learning With Errors problem). Even in pure mathematics, the Gaussian appears as the extremal function in the entropy-maximization and Fourier-uncertainty problems, and its self-reproducing property under Fourier transform makes it central to harmonic analysis.
:::quiz[Why does the "boundary term" vanish in the variance computation?]
- Because itself equals zero at
- Because exponential decay of dominates the linear growth of ✓
- Because the standard normal is symmetric
- Because
Ask the tutor: How does the Central Limit Theorem explain why so many real-world quantities end up approximately Normal?
A continuous random variable is said to have a Normal distribution with parameters and , written , if its probability density function is
When and , is called a standard normal random variable, conventionally denoted , with density
and cumulative distribution function
Intuition
Picture the density as a bell centered at : it is symmetric about , rises to a single peak there, and decays to zero in both directions, faster than any exponential because of the quadratic exponent. The parameter slides the whole curve left or right without changing its shape; the parameter stretches or compresses it horizontally while the total area under the curve — always exactly — is preserved by a compensating change in height. This is precisely what standardization exploits: subtracting recenters the curve at , and dividing by rescales it back to the one universal shape . The reason Gaussians appear everywhere is intimately tied to the Central Limit Theorem (treated elsewhere): sums of many small, roughly independent contributions — measurement errors, thermal fluctuations, aggregated individual traits — wash out their individual quirks and converge to this one shape. The Normal distribution is thus less an assumption about the world and more a mathematical attractor that many different mechanisms funnel into.
Let and be fixed constants, and define
Then: 1. (Validity) is a valid probability density function: for all , and . 2. (Moments) If has density , then and . 3. (Standardization) If and , then . Consequently, for all ,
We prove each part in turn. Throughout, denotes the standard normal density.
Step 1: The Gaussian integral. We first show
Since the integrand is positive, , and it suffices to compute . Using Fubini's theorem (justified because the integrand is nonnegative and, as shown below, has finite integral), write as a double integral and convert to polar coordinates , , with Jacobian :
The inner integral is elementary: substituting , ,
Hence , and since , we conclude .
Step 2: Validity for the standard normal. Positivity of is immediate since the exponential function is always positive. By Step 1,
So is a valid density.
Step 3: Validity for general . For the general density , substitute , so and . As ranges over , so does (since makes this an increasing linear bijection). Then
using Step 2. Positivity of is clear since it is a positive constant times an exponential. This proves part 1.
Step 4: The mean of the standard normal is . Let have density . Provided the integral converges absolutely (verified below), we can compute
First check absolute convergence: by symmetry of , and
using the substitution . So the mean exists. The integrand is an odd function of (since is even), so for any , by pairing with . Taking and using the absolute convergence just shown to justify the limit,
Step 5: The variance of the standard normal is . We compute using integration by parts. Note that , so we write and integrate by parts with , :
The boundary term vanishes because as (exponential decay dominates linear growth). The remaining integral is by Step 2. Hence . Since by Step 4,
Step 6: Standardization. Let with density as in the theorem statement, and define . We find the density of using the CDF technique for a strictly increasing differentiable transformation. Since , the map is strictly increasing, so for any :
Differentiating both sides with respect to using the chain rule (valid since is differentiable with derivative , the continuous density):
Thus , proving the first claim of part 3.
Step 7: Moments of general . From Step 6, with . By linearity of expectation (a general fact about expectations of continuous random variables, not dependent on normality),
using Step 4. By the scaling property of variance, for constants (a direct consequence of the definition of variance, independent of the distribution of ), so
using Step 5. This proves part 2.
Step 8: Probability formula. Finally, since by Step 6,
where the middle equality is the definition of a CDF applied to the event (an equivalence following from monotonicity of ), and continuity of ensures no point-mass issues at the endpoints. This completes the proof of part 3, and the theorem.
∎Worked examples
Example 1 (asymmetric interval, non-round parameters). Suppose the length (in mm) of a manufactured bolt is modeled as with and . A bolt is acceptable if . What is the probability a randomly selected bolt is acceptable? By the Standardization Theorem,
We compute each term. From standard normal tables (or numerical integration of ), and , so by symmetry . Interpolating more precisely: , . Then
So roughly 85.6% of bolts are acceptable. Notice the interval is not symmetric about (it extends below and above), which is exactly why both standardized bounds ( and ) had to be handled separately rather than exploiting symmetry to halve the work — a deliberate check that the computation does not silently rely on a symmetric special case.
Example 2 (working backward to find a cutoff — inverse problem). A university sets admission so that only the top of applicants by score are admitted. Scores are modeled as (so , ). Find the minimum score required for admission. We need , i.e., . Standardizing,
From standard normal tables, (the value with ). Solving,
So an applicant needs a score of approximately 615.3 or higher to be admitted. This example exercises the inverse direction of standardization — going from a probability to a cutoff value, via rather than — which is the operation underlying critical values in hypothesis testing.
The theorem requires as a standing hypothesis. Suppose instead we attempted to define a "Normal density" with , i.e.
This expression is not even well-defined: division by zero occurs both in the normalizing constant and in the exponent. There is no way to interpret this as a function , so fails to be a density before we can even ask whether it integrates to . (In the limiting sense, as with fixed, the family concentrates all its mass at the single point , converging weakly to the Dirac point mass — but is not a probability density function with respect to Lebesgue measure, since a discrete point mass cannot be represented by any integrable function.) If instead we allowed , the exponent would be positive for , and the function would grow without bound as rather than decay. Concretely, with (so the exponent becomes ), the "density" satisfies , since the integrand is unbounded and increasing away from . Hence is not integrable at all, let alone normalizable to total mass . This shows the hypothesis is not a technical nicety but is load-bearing: without it, the object in the definition ceases to be a probability density altogether.
Scope, edge cases & common traps
| Situation | What to watch for |
|---|---|
| with very small | concentrates tightly near ; density spikes high, but total area is still |
| Standardizing a sum of Normals | Requires independence (or a joint Normal/multivariate structure) to conclude the sum is Normal; simple standardization of alone does not need independence |
| Reading off a table | Use ; tables often list only |
| Confusing and | The notation names the variance, not the standard deviation, as the second parameter |
| Applying to a non-standardized variable | is only the CDF of ; unless is already standard normal |
The mathematician behind it
Origins · 1 minCarl Friedrich Gauss and how this was discovered
Gauss developed the theory of errors in astronomical and geodetic measurement, deriving the normal density as the distribution of measurement errors under natural assumptions about how errors combine, in his 1809 work on the motion of celestial bodies. Although Abraham de Moivre had already found the same curve decades earlier as an approximation to the binomial distribution, and Pierre-Simon Laplace developed the general limiting theory later known as the Central Limit Theorem, Gauss's application to observational error theory was so influential in astronomy and physics that the curve became permanently attached to his name.
Why does the Normal distribution have exactly two parameters, and ?
The two parameters correspond exactly to the two degrees of freedom in the family: location and scale. Every member of the Normal family is generated from the single standard curve by the affine transformation , so specifying where the curve is centered () and how spread out it is () completely determines the shape. No additional shape parameters (like skewness or kurtosis controls) exist, because the Normal density's functional form is rigidly fixed as a Gaussian exponential — unlike families such as the Beta or Gamma distributions, which have separate shape parameters beyond location and scale.
Is there a closed-form expression for ?
No elementary closed form exists for ; it can be shown (though the proof is a result in differential algebra, not covered here) that has no antiderivative expressible in terms of elementary functions. In practice, is evaluated using tables, numerical integration, or its close relative the error function , via the identity . Statistical software and calculators compute using efficient series or continued-fraction approximations rather than any elementary formula.
Why is the sum of two independent Normal random variables also Normal?
This closure-under-addition property is a genuinely special feature of the Normal family, not shared by most distributions. If and are independent, then . The cleanest proof uses moment generating functions: the MGF of a sum of independent variables is the product of their MGFs, and the Normal MGF has a special exponential-quadratic form whose products stay within the same family. This is exactly the kind of computation that Moment Generating Functions are designed to make routine.
Can a Normal random variable take negative values, even if it models something like height or weight that must be positive?
Yes — technically, the support of any is all of , so it always assigns strictly positive (if tiny) probability to negative values, which can be a modeling flaw for inherently positive quantities. In practice, if is many standard deviations above zero (e.g., ), this probability is astronomically small and the Normal model remains an excellent practical approximation. When the discrepancy matters, analysts often switch to a log-normal or truncated normal model instead.
What is the difference between the standard normal density and the standard normal CDF ?
is the density function: represents (heuristically) the probability of falling in an infinitesimal interval near , and itself can exceed in value (its peak is ). is the cumulative distribution function, always between and , representing . They are related by differentiation: , by the Fundamental Theorem of Calculus, since is continuous.
Does standardization change the shape of the distribution, or just its location and scale?
Standardization is precisely an affine (linear plus shift) transformation, and the proof in this article shows it produces exactly the standard normal density with no distortion of shape — the Jacobian factor from the change of variables exactly cancels the already present in . This is special to Normal random variables having this rigid two-parameter affine family structure; for other distributions, an affine transformation can produce a variable outside the original family entirely.
Why is (the inverse standard normal CDF, or "quantile function") important in practice?
answers the inverse question: given a target probability , what standardized cutoff satisfies ? This is exactly the computation behind statistical critical values (like for a two-sided 95% confidence interval, since ) and behind the admissions-cutoff calculation in Example 2 above. has no elementary closed form either, and is computed via table lookup, numerical root-finding on , or specialized rational approximations in software.