Variance and Moments

How spread out is a random quantity around its average value? A weather forecaster who says "the expected rainfall is 2 inches" has told you almost nothing. You don't know whether to expect a drizzle or a flood. You need a second number that measures the typical deviation from that average. This is exactly the gap that variance fills. It is one of the most consequential ideas in all of probability and statistics. It underlies error bars, confidence intervals, risk management, and the entire theory of statistical estimation. The idea builds directly on Expectation. Instead of asking only for the average value E[X]E[X] of a random variable, we ask for something more. We ask for the average squared distance from that value, E[(X−E[X])2]E[(X-E[X])^2]. Squaring is not arbitrary. It penalizes large deviations more than small ones. It treats positive and negative deviations symmetrically. Crucially, it also makes the resulting quantity algebraically tractable. It decomposes nicely for sums of variables. Variance is the second in an infinite hierarchy of numbers called moments. Each one captures progressively finer features of a distribution's shape. The third moment detects asymmetry (skewness). The fourth detects tail-heaviness (kurtosis), and so on. This article gives the rigorous definition of variance and standard deviation. It proves the core structural facts: the shortcut formula, scaling and shift rules, additivity for independent variables. It also situates variance within the broader family of moments. We also prove the two workhorse inequalities of the subject — Markov's and Chebyshev's. These convert "a bounded second moment" into quantitative control over how far a random variable can stray from its mean. By the end you will be able to compute variances confidently. You will know exactly which hypotheses each identity requires (independence is not optional for additivity!). You will also see how this single number — E[(X−μ)2]E[(X-\mu)^2] — sets up the machinery of Moment Generating Functions. These package all moments into a single analytic object.

Key takeaways
  • Variance Var(X)=E[(X−μ)2]\mathrm{Var}(X) = E[(X-\mu)^2] measures average squared deviation from the mean μ=E[X]\mu = E[X].
  • Standard deviation σ=Var(X)\sigma = \sqrt{\mathrm{Var}(X)} restores the original units, making it directly interpretable.
  • The shortcut formula Var(X)=E[X2]−(E[X])2\mathrm{Var}(X) = E[X^2] - (E[X])^2 is the standard computational tool.
  • Variance is not linear: Var(aX+b)=a2Var(X)\mathrm{Var}(aX+b) = a^2\mathrm{Var}(X), and additivity over sums needs independence (or zero covariance).
  • Higher moments — third (skewness), fourth (kurtosis) — refine the picture of a distribution's shape beyond mean and spread.
  • Key idea: variance quantifies dispersion by averaging squared distance from the mean, and this single number drives concentration inequalities like Chebyshev's.

Where this lives today

Variance is the workhorse of quantitative uncertainty across nearly every applied field.

  • Statistics. The sample variance is the basic estimator of spread, feeding directly into

confidence intervals, hypothesis tests (t-tests, ANOVA), and regression diagnostics (residual variance, R2R^2).

  • Finance. Portfolio variance (and covariance between assets) is the central object of

Markowitz mean-variance portfolio theory, where "risk" is literally defined as standard deviation of returns.

  • Physics and engineering. Measurement uncertainty is reported as standard deviation;

signal processing quantifies noise power as variance, and Chebyshev-type bounds justify error-tolerance guarantees without assuming a specific noise distribution.

  • Machine learning. The bias–variance decomposition of prediction error explains

overfitting versus underfitting, and variance reduction techniques (e.g. in Monte Carlo estimation) directly exploit the additivity and scaling rules proved here.

  • Cryptography and randomized algorithms. Concentration inequalities built on variance

(Chebyshev, and its sharper descendants like Chernoff bounds) certify that randomized algorithms behave predictably with high probability.

  • Foreshadowing. Packaging all moments E[Xn]E[X^n] simultaneously into one generating

function is exactly the idea behind Moment Generating Functions, where variance reappears as the second derivative (at 00) of log⁡MX(t)\log M_X(t) — a preview of how this article's toolkit scales up.

Ask the tutor: How does variance appear as a derivative of the moment generating function, and why does that make it easy to compute for sums of independent variables?

DefinitionMoments, Variance, and Standard Deviation

Let XX be a random variable on a probability space (Ω,F,P)(\Omega, \mathcal F, P).

nn-th moment. For a positive integer nn, the nn-th moment of XX is

μn′=E[Xn],\mu_n' = E[X^n],

defined whenever E[∣X∣n]<∞E[|X|^n] < \infty. The first moment μ1′=E[X]\mu_1' = E[X] is the mean, often written μ\mu.

nn-th central moment. Assuming E[∣X∣n]<∞E[|X|^n]<\infty, the nn-th central moment is

μn=E[(X−μ)n].\mu_n = E\big[(X-\mu)^n\big].

Variance. The variance of XX, denoted Var(X)\mathrm{Var}(X) or σX2\sigma_X^2, is the second central moment:

Var(X)=E[(X−μ)2],\mathrm{Var}(X) = E\big[(X-\mu)^2\big],

provided E[X2]<∞E[X^2] < \infty (so that μ\mu and this expectation both exist and are finite).

Standard deviation. The standard deviation of XX is

σX=Var(X),\sigma_X = \sqrt{\mathrm{Var}(X)},

the nonnegative square root, expressed in the same units as XX itself.

Skewness and kurtosis. When the relevant moments are finite and σX>0\sigma_X > 0, define the (standardized) skewness and excess kurtosis

γ1=E[(X−μ)3]σX3,γ2=E[(X−μ)4]σX4−3.\gamma_1 = \frac{E[(X-\mu)^3]}{\sigma_X^3}, \qquad \gamma_2 = \frac{E[(X-\mu)^4]}{\sigma_X^4} - 3.

All of these definitions apply verbatim to discrete random variables (sums replacing integrals in the underlying expectations) and to continuous random variables with density fXf_X, where for instance Var(X)=∫−∞∞(x−μ)2fX(x) dx\mathrm{Var}(X) = \int_{-\infty}^{\infty}(x-\mu)^2 f_X(x)\,dx.

Intuition

Think of E[X]E[X] as the balance point of a distribution of mass along a line — if you cut a cardboard cutout shaped like the density and tried to balance it on a pin, the pin would go under μ\mu. Variance then asks: on average, how far is the mass from that balance point, counted in squared distance? A distribution with all its mass piled near μ\mu has small variance; a distribution with mass spread far out (even if still balanced at μ\mu) has large variance. Squaring the deviation X−μX - \mu serves two purposes. First, it eliminates the sign problem: E[X−μ]=0E[X-\mu] = 0 always, so raw deviations cannot measure spread (positive and negative deviations cancel). Second, squaring emphasizes large deviations disproportionately — a deviation of 1010 contributes 100100 times more than a deviation of 11 — which turns out to be exactly the right weighting to make variance behave beautifully under sums of independent variables, as the additivity theorem below shows. The standard deviation σ\sigma undoes the "squared units" problem: if XX is measured in meters, Var(X)\mathrm{Var}(X) is in meters², which is hard to interpret, but σ\sigma is back in meters and can be compared directly to μ\mu or to observed values of XX.

Figure
Two densities with equal mean but different spread: variance quantifies the difference
x μ = 0 σ = 1 (tight) σ = 3 (spread) 1σ 3σ
TheoremBasic Properties of Variance

Let X,YX, Y be random variables with E[X2]<∞E[X^2] < \infty and E[Y2]<∞E[Y^2] < \infty, and let a,b∈Ra, b \in \mathbb R be constants. Then: 1. (Nonnegativity) Var(X)≥0\mathrm{Var}(X) \ge 0, with equality if and only if X=μX = \mu almost surely (i.e. XX is constant with probability 11). 2. (Shortcut formula) Var(X)=E[X2]−(E[X])2\mathrm{Var}(X) = E[X^2] - (E[X])^2. 3. (Affine transformation) Var(aX+b)=a2Var(X)\mathrm{Var}(aX+b) = a^2 \mathrm{Var}(X). 4. (Bilinear expansion) Var(X+Y)=Var(X)+Var(Y)+2 Cov(X,Y)\mathrm{Var}(X+Y) = \mathrm{Var}(X) + \mathrm{Var}(Y) + 2\,\mathrm{Cov}(X,Y), where Cov(X,Y)=E[(X−EX)(Y−EY)]\mathrm{Cov}(X,Y) = E[(X-E X)(Y - E Y)]. 5. (Additivity under independence) If XX and YY are independent, then Var(X+Y)=Var(X)+Var(Y)\mathrm{Var}(X+Y) = \mathrm{Var}(X) + \mathrm{Var}(Y).

Proof

We prove each part using only the linearity of expectation for Expectation and the definitions above; no part depends on later parts in a circular way. Throughout, μ=E[X]\mu = E[X] and ν=E[Y]\nu = E[Y], both finite since E[X2],E[Y2]<∞E[X^2], E[Y^2] < \infty implies (by Cauchy–Schwarz, E[∣X∣]≤E[X2]E[|X|] \le \sqrt{E[X^2]}) that E[∣X∣]<∞E[|X|]<\infty, and similarly for YY.

(1) Nonnegativity. The random variable (X−μ)2(X-\mu)^2 is nonnegative for every outcome, so its expectation is nonnegative: Var(X)=E[(X−μ)2]≥0\mathrm{Var}(X) = E[(X-\mu)^2] \ge 0. If X=μX = \mu almost surely, then (X−μ)2=0(X-\mu)^2 = 0 almost surely, so its expectation is 00. Conversely, suppose Var(X)=0\mathrm{Var}(X) = 0. Since (X−μ)2≥0(X-\mu)^2 \ge 0 and E[(X−μ)2]=0E[(X-\mu)^2] = 0, standard measure-theoretic fact ("nonnegative random variable with zero expectation is zero almost surely") gives (X−μ)2=0(X-\mu)^2 = 0 almost surely, hence X=μX = \mu almost surely. (This underlying fact follows from Markov's inequality applied to (X−μ)2(X-\mu)^2, proved independently below, so there is no circularity: for any ε>0\varepsilon>0, P((X−μ)2≥ε)≤E[(X−μ)2]/ε=0P((X-\mu)^2 \ge \varepsilon) \le E[(X-\mu)^2]/\varepsilon = 0; letting ε→0\varepsilon \to 0 along a sequence and using continuity of measure gives P(X≠μ)=0P(X \ne \mu) = 0.)

(2) Shortcut formula. Expand the square and use linearity of expectation, valid because E[X2]<∞E[X^2]<\infty guarantees all terms below are finite:

Var(X)=E[(X−μ)2]=E[X2−2μX+μ2]=E[X2]−2μE[X]+μ2=E[X2]−2μ2+μ2=E[X2]−μ2.\mathrm{Var}(X) = E[(X-\mu)^2] = E[X^2 - 2\mu X + \mu^2] = E[X^2] - 2\mu E[X] + \mu^2 = E[X^2] - 2\mu^2 + \mu^2 = E[X^2] - \mu^2.

Since μ=E[X]\mu = E[X], this is E[X2]−(E[X])2E[X^2] - (E[X])^2.

(3) Affine transformation. Let Z=aX+bZ = aX+b. Then E[Z]=aE[X]+b=aμ+bE[Z] = aE[X]+b = a\mu+b, so

Z−E[Z]=(aX+b)−(aμ+b)=a(X−μ).Z - E[Z] = (aX+b) - (a\mu+b) = a(X-\mu).

Hence

Var(aX+b)=E[(a(X−μ))2]=a2E[(X−μ)2]=a2 Var(X).\mathrm{Var}(aX+b) = E\big[(a(X-\mu))^2\big] = a^2 E[(X-\mu)^2] = a^2\,\mathrm{Var}(X).

(4) Bilinear expansion. Let S=X+YS = X+Y, so E[S]=μ+νE[S] = \mu+\nu by linearity of expectation, and

S−E[S]=(X−μ)+(Y−ν).S - E[S] = (X-\mu) + (Y-\nu).

Squaring,

(S−E[S])2=(X−μ)2+(Y−ν)2+2(X−μ)(Y−ν).(S-E[S])^2 = (X-\mu)^2 + (Y-\nu)^2 + 2(X-\mu)(Y-\nu).

Taking expectations (linearity, and each term has finite expectation since E[X2],E[Y2]<∞E[X^2],E[Y^2]<\infty and Cauchy–Schwarz bounds the cross term: E[∣(X−μ)(Y−ν)∣]≤Var(X)Var(Y)<∞E[|(X-\mu)(Y-\nu)|] \le \sqrt{\mathrm{Var}(X)\mathrm{Var}(Y)} <\infty):

Var(X+Y)=Var(X)+Var(Y)+2E[(X−μ)(Y−ν)]=Var(X)+Var(Y)+2 Cov(X,Y).\mathrm{Var}(X+Y) = \mathrm{Var}(X) + \mathrm{Var}(Y) + 2E[(X-\mu)(Y-\nu)] = \mathrm{Var}(X)+\mathrm{Var}(Y)+2\,\mathrm{Cov}(X,Y).

(5) Additivity under independence. If X,YX,Y are independent, then so are X−μX-\mu and Y−νY-\nu (a measurable function of XX is independent of any measurable function of YY when X⊥YX \perp Y). For independent random variables U,VU,V with finite second moments, E[UV]=E[U]E[V]E[UV] = E[U]E[V] — this is the standard factorization property of expectation under independence, applied here with U=X−μU = X-\mu, V=Y−νV=Y-\nu. Since E[X−μ]=0E[X-\mu]=0 and E[Y−ν]=0E[Y-\nu]=0,

Cov(X,Y)=E[(X−μ)(Y−ν)]=E[X−μ] E[Y−ν]=0.\mathrm{Cov}(X,Y) = E[(X-\mu)(Y-\nu)] = E[X-\mu]\,E[Y-\nu] = 0.

Substituting into part (4) gives Var(X+Y)=Var(X)+Var(Y)\mathrm{Var}(X+Y) = \mathrm{Var}(X)+\mathrm{Var}(Y). ■\blacksquare

∎
CorollarySums of independent variables

If X1,…,XnX_1, \dots, X_n are pairwise independent with finite second moments, then

Var(X1+⋯+Xn)=∑i=1nVar(Xi).\mathrm{Var}(X_1 + \cdots + X_n) = \sum_{i=1}^n \mathrm{Var}(X_i).

This follows from part (5) of the theorem by induction on nn: writing Sk=X1+⋯+XkS_{k} = X_1+\cdots+X_k, one checks Sk−1S_{k-1} and XkX_k are independent (a consequence of pairwise independence combined with the fact that Cov\mathrm{Cov} only needs pairwise vanishing, since the bilinear expansion of Var(Sk−1+Xk)\mathrm{Var}(S_{k-1}+X_k) only involves the single covariance term Cov(Sk−1,Xk)=∑i<kCov(Xi,Xk)=0\mathrm{Cov}(S_{k-1},X_k) = \sum_{i<k}\mathrm{Cov}(X_i,X_k) = 0), and the base case n=1n=1 is trivial.

LemmaMarkov's Inequality

If Y≥0Y \ge 0 is a random variable with E[Y]<∞E[Y] < \infty, then for every a>0a > 0,

P(Y≥a)≤E[Y]a.P(Y \ge a) \le \frac{E[Y]}{a}.
Proof

Let A={Y≥a}A = \{Y \ge a\} and let 1A\mathbf 1_A be its indicator. Since Y≥0Y \ge 0 everywhere, and on AA we have Y≥aY \ge a, we get the pointwise inequality

Y≥a 1AY \ge a\,\mathbf 1_A

valid on all of Ω\Omega: on AA, the right side is a≤Ya \le Y; off AA, the right side is 0≤Y0 \le Y since Y≥0Y \ge 0. Taking expectations, which preserves this inequality since expectation is monotone,

E[Y]≥a E[1A]=a P(A)=a P(Y≥a).E[Y] \ge a\,E[\mathbf 1_A] = a\,P(A) = a\,P(Y \ge a).

Dividing by a>0a>0 gives the result. ■\blacksquare

∎
TheoremChebyshev's Inequality

Let XX be a random variable with mean μ=E[X]\mu = E[X] and finite variance σ2=Var(X)\sigma^2 = \mathrm{Var}(X). Then for every k>0k > 0,

P(∣X−μ∣≥k)≤σ2k2.P(|X - \mu| \ge k) \le \frac{\sigma^2}{k^2}.
Proof

Apply Markov's Inequality (Lemma above) to the nonnegative random variable Y=(X−μ)2Y = (X-\mu)^2 with threshold a=k2>0a = k^2 > 0:

P((X−μ)2≥k2)≤E[(X−μ)2]k2=σ2k2.P\big((X-\mu)^2 \ge k^2\big) \le \frac{E[(X-\mu)^2]}{k^2} = \frac{\sigma^2}{k^2}.

Finally observe the event equality

{(X−μ)2≥k2}={∣X−μ∣≥k},\{(X-\mu)^2 \ge k^2\} = \{|X-\mu| \ge k\},

which holds because for real numbers, t2≥k2  ⟺  ∣t∣≥kt^2 \ge k^2 \iff |t| \ge k (both sides squaring the same nonnegative quantity kk). Hence P(∣X−μ∣≥k)=P((X−μ)2≥k2)≤σ2/k2P(|X-\mu|\ge k) = P((X-\mu)^2 \ge k^2) \le \sigma^2/k^2. ■\blacksquare

∎

Worked examples

Example 1 — A biased, asymmetric die. Let XX be the outcome of a roll of a loaded six-sided die with probability mass function

P(X=1)=0.05,  P(X=2)=0.10,  P(X=3)=0.15,  P(X=4)=0.20,  P(X=5)=0.20,  P(X=6)=0.30.P(X=1) = 0.05,\; P(X=2)=0.10,\; P(X=3)=0.15,\; P(X=4)=0.20,\; P(X=5)=0.20,\; P(X=6)=0.30.

(Check: 0.05+0.10+0.15+0.20+0.20+0.30=1.000.05+0.10+0.15+0.20+0.20+0.30 = 1.00. ✓) Mean.

μ=E[X]=1(0.05)+2(0.10)+3(0.15)+4(0.20)+5(0.20)+6(0.30).\mu = E[X] = 1(0.05)+2(0.10)+3(0.15)+4(0.20)+5(0.20)+6(0.30).

Compute term by term: 0.05,0.20,0.45,0.80,1.00,1.800.05, 0.20, 0.45, 0.80, 1.00, 1.80. Summing: 0.05+0.20=0.250.05+0.20 = 0.25; +0.45=0.70+0.45 = 0.70; +0.80=1.50+0.80=1.50; +1.00=2.50+1.00=2.50; +1.80=4.30+1.80=4.30. So μ=4.30\mu = 4.30. Second moment.

E[X2]=12(0.05)+22(0.10)+32(0.15)+42(0.20)+52(0.20)+62(0.30).E[X^2] = 1^2(0.05)+2^2(0.10)+3^2(0.15)+4^2(0.20)+5^2(0.20)+6^2(0.30).

Term by term: 1(0.05)=0.051(0.05)=0.05; 4(0.10)=0.404(0.10)=0.40; 9(0.15)=1.359(0.15)=1.35; 16(0.20)=3.2016(0.20)=3.20; 25(0.20)=5.0025(0.20)=5.00; 36(0.30)=10.8036(0.30)=10.80. Summing: 0.05+0.40=0.450.05+0.40=0.45; +1.35=1.80+1.35=1.80; +3.20=5.00+3.20=5.00; +5.00=10.00+5.00=10.00; +10.80=20.80+10.80 = 20.80. So E[X2]=20.80E[X^2] = 20.80. Variance via the shortcut formula (Theorem, part 2).

Var(X)=E[X2]−μ2=20.80−(4.30)2=20.80−18.49=2.31.\mathrm{Var}(X) = E[X^2] - \mu^2 = 20.80 - (4.30)^2 = 20.80 - 18.49 = 2.31.

Standard deviation.

σX=2.31≈1.5199.\sigma_X = \sqrt{2.31} \approx 1.5199.

Sanity check via Chebyshev. With k=2k = 2, Chebyshev's inequality predicts

P(∣X−4.30∣≥2)≤2.314=0.5775.P(|X - 4.30| \ge 2) \le \frac{2.31}{4} = 0.5775.

Directly: ∣X−4.30∣≥2|X-4.30|\ge 2 means X≤2.30X \le 2.30 or X≥6.30X \ge 6.30, i.e. X∈{1,2}X \in \{1,2\} (since X≤6X\le 6). This has probability 0.05+0.10=0.150.05+0.10=0.15, comfortably below the Chebyshev bound 0.57750.5775 — consistent with the theorem (which gives an upper bound, not an exact value).

Example 2 — An exponential random variable and a linear rescaling. Let XX have the exponential distribution with rate λ=0.4\lambda = 0.4, i.e. density fX(x)=0.4 e−0.4xf_X(x) = 0.4\,e^{-0.4x} for x≥0x \ge 0. Define Y=5X−3Y = 5X - 3 (say YY represents a rescaled and shifted measurement, such as a temperature conversion of a failure time). Moments of XX. For an exponential(λ\lambda) variable it is a standard computation (via integration by parts, or the Gamma function identity ∫0∞xne−λxdx=n!/λn+1\int_0^\infty x^n e^{-\lambda x}dx = n!/\lambda^{n+1}) that

E[X]=1λ,E[X2]=2λ2.E[X] = \frac{1}{\lambda}, \qquad E[X^2] = \frac{2}{\lambda^2}.

With λ=0.4\lambda = 0.4:

E[X]=10.4=2.5,E[X2]=20.16=12.5.E[X] = \frac{1}{0.4} = 2.5, \qquad E[X^2] = \frac{2}{0.16} = 12.5.

Variance of XX.

Var(X)=E[X2]−(E[X])2=12.5−(2.5)2=12.5−6.25=6.25.\mathrm{Var}(X) = E[X^2] - (E[X])^2 = 12.5 - (2.5)^2 = 12.5 - 6.25 = 6.25.

(This matches the known general formula Var(X)=1/λ2=1/0.16=6.25\mathrm{Var}(X) = 1/\lambda^2 = 1/0.16 = 6.25.) Variance and mean of Y=5X−3Y = 5X-3. By linearity of expectation, E[Y]=5E[X]−3=5(2.5)−3=12.5−3=9.5E[Y] = 5E[X] - 3 = 5(2.5) - 3 = 12.5 - 3 = 9.5. By the affine-transformation rule (Theorem, part 3), the additive shift −3-3 does not affect variance, but the scale factor 55 enters squared:

Var(Y)=52 Var(X)=25×6.25=156.25.\mathrm{Var}(Y) = 5^2 \,\mathrm{Var}(X) = 25 \times 6.25 = 156.25.

So σY=156.25=12.5\sigma_Y = \sqrt{156.25} = 12.5. Cross-check by direct expansion. Alternatively, Y−E[Y]=5X−3−9.5=5(X−2.5)=5(X−E[X])Y - E[Y] = 5X - 3 - 9.5 = 5(X-2.5) = 5(X - E[X]), so (Y−E[Y])2=25(X−E[X])2(Y-E[Y])^2 = 25(X-E[X])^2 pointwise, and taking expectations reproduces Var(Y)=25 Var(X)=156.25\mathrm{Var}(Y) = 25\,\mathrm{Var}(X) = 156.25 exactly, confirming the theorem's formula without any shortcut-formula bookkeeping. Skewness sanity check. The exponential distribution is known to have skewness γ1=2\gamma_1 = 2 for every rate λ\lambda; since skewness is scale invariant and shift invariant (as a ratio of an odd central moment to σ3\sigma^3, unaffected by adding a constant, and both numerator and denominator scale identically under a positive scale factor — here λ=5>0\lambda = 5 >0 preserves sign), Y=5X−3Y = 5X-3 has the same skewness γ1=2\gamma_1 = 2 as XX, illustrating that skewness (unlike variance) is a shape descriptor invariant under this class of transformations.

CounterexampleAdditivity of variance fails without independence

The additivity formula Var(X+Y)=Var(X)+Var(Y)\mathrm{Var}(X+Y) = \mathrm{Var}(X) + \mathrm{Var}(Y) in part (5) of the Theorem requires independence (or, more generally, Cov(X,Y)=0\mathrm{Cov}(X,Y) = 0). Here is a concrete case where it is dropped and the conclusion fails. Let XX be a random variable taking values −1,0,1-1, 0, 1 each with probability 1/31/3, and define Y=X2Y = X^2. Then XX and YY are manifestly dependent — knowing Y=1Y=1 tells you X∈{−1,1}X \in \{-1,1\}, changing the conditional distribution of XX from its marginal. Compute the pieces. First, YY takes the value 11 with probability 2/32/3 (from X=±1X=\pm1) and 00 with probability 1/31/3 (from X=0X=0). So:

E[X]=13(−1)+13(0)+13(1)=0,E[X2]=13(1)+13(0)+13(1)=23,E[X] = \tfrac13(-1) + \tfrac13(0) + \tfrac13(1) = 0, \qquad E[X^2] = \tfrac13(1)+\tfrac13(0)+\tfrac13(1) = \tfrac23,

so Var(X)=E[X2]−02=2/3\mathrm{Var}(X) = E[X^2] - 0^2 = 2/3. For Y=X2∈{0,1}Y = X^2 \in\{0,1\} with P(Y=1)=2/3P(Y=1)=2/3, P(Y=0)=1/3P(Y=0)=1/3:

E[Y]=23,E[Y2]=E[X4]=13(1)+13(0)+13(1)=23,E[Y] = \tfrac23, \qquad E[Y^2] = E[X^4] = \tfrac13(1)+\tfrac13(0)+\tfrac13(1) = \tfrac23,

so Var(Y)=23−(23)2=23−49=69−49=29\mathrm{Var}(Y) = \tfrac23 - \left(\tfrac23\right)^2 = \tfrac23 - \tfrac49 = \tfrac{6}{9}-\tfrac{4}{9} = \tfrac29. Now consider S=X+Y=X+X2S = X + Y = X + X^2. Its possible values: if X=−1X=-1, S=−1+1=0S = -1+1=0; if X=0X=0, S=0+0=0S=0+0=0; if X=1X=1, S=1+1=2S=1+1=2. So SS takes value 00 with probability 2/32/3 (from X=−1X=-1 or X=0X=0) and value 22 with probability 1/31/3 (from X=1X=1).

E[S]=23(0)+13(2)=23,E[S2]=23(0)+13(4)=43,E[S] = \tfrac23(0) + \tfrac13(2) = \tfrac23, \qquad E[S^2] = \tfrac23(0) + \tfrac13(4) = \tfrac43,

so

Var(S)=43−(23)2=43−49=129−49=89.\mathrm{Var}(S) = \tfrac43 - \left(\tfrac23\right)^2 = \tfrac43 - \tfrac49 = \tfrac{12}{9}-\tfrac49 = \tfrac{8}{9}.

Compare: Var(X)+Var(Y)=23+29=69+29=89\mathrm{Var}(X) + \mathrm{Var}(Y) = \tfrac23 + \tfrac29 = \tfrac69+\tfrac29 = \tfrac89. Remarkably the numbers match here — so let's instead directly check whether the covariance is zero, since that is the actual mechanism. Compute

Cov(X,Y)=E[XY]−E[X]E[Y]=E[X3]−0.\mathrm{Cov}(X,Y) = E[XY] - E[X]E[Y] = E[X^3] - 0.

E[X3]=13(−1)+13(0)+13(1)=0E[X^3] = \tfrac13(-1)+\tfrac13(0)+\tfrac13(1) = 0, so in this particular symmetric example the covariance happens to vanish even though X,YX,Y are dependent — a good reminder that zero covariance does not imply independence, and this example's coincidental agreement is not a proof that dependence is harmless. To see the additivity formula genuinely fail, perturb the example to break the symmetry: let XX take values −1,0,2-1,0,2 with probabilities 1/2,1/4,1/41/2, 1/4, 1/4 and again set Y=X2Y=X^2. Then:

E[X]=12(−1)+14(0)+14(2)=−12+12=0,E[X] = \tfrac12(-1)+\tfrac14(0)+\tfrac14(2) = -\tfrac12+\tfrac12 = 0,
E[X2]=12(1)+14(0)+14(4)=12+1=32,Var(X)=32.E[X^2] = \tfrac12(1)+\tfrac14(0)+\tfrac14(4) = \tfrac12+1 = \tfrac32,\quad \mathrm{Var}(X)=\tfrac32.

Y=X2Y=X^2 takes value 11 w.p. 1/21/2, 00 w.p. 1/41/4, 44 w.p. 1/41/4:

E[Y]=32,E[Y2]=E[X4]=12(1)+14(0)+14(16)=12+4=92,E[Y]=\tfrac32,\quad E[Y^2]=E[X^4]=\tfrac12(1)+\tfrac14(0)+\tfrac14(16)=\tfrac12+4=\tfrac92,
Var(Y)=92−(32)2=92−94=94.\mathrm{Var}(Y) = \tfrac92 - \left(\tfrac32\right)^2 = \tfrac92-\tfrac94=\tfrac94.

Now Var(X)+Var(Y)=32+94=64+94=154\mathrm{Var}(X)+\mathrm{Var}(Y) = \tfrac32+\tfrac94 = \tfrac64+\tfrac94=\tfrac{15}{4}. For S=X+Y=X+X2S=X+Y=X+X^2: at X=−1X=-1, S=−1+1=0S=-1+1=0; at X=0X=0, S=0S=0; at X=2X=2, S=2+4=6S=2+4=6. So S=0S=0 w.p. 3/43/4 and S=6S=6 w.p. 1/41/4.

E[S]=34(0)+14(6)=64=32,E[S2]=34(0)+14(36)=9,E[S] = \tfrac34(0)+\tfrac14(6) = \tfrac64=\tfrac32, \qquad E[S^2] = \tfrac34(0)+\tfrac14(36)=9,
Var(S)=9−(32)2=9−94=274.\mathrm{Var}(S) = 9 - \left(\tfrac32\right)^2 = 9-\tfrac94 = \tfrac{27}{4}.

Now compare Var(S)=27/4=6.75\mathrm{Var}(S) = 27/4 = 6.75 against Var(X)+Var(Y)=15/4=3.75\mathrm{Var}(X)+\mathrm{Var}(Y) = 15/4 = 3.75: they clearly disagree. This confirms that dropping independence breaks additivity — the "missing" 2 Cov(X,Y)2\,\mathrm{Cov}(X,Y) term makes up the difference exactly as part (4) of the Theorem predicts, since here Cov(X,Y)=E[X3]−0=12(−1)+14(0)+14(8)=−12+2=32\mathrm{Cov}(X,Y) = E[X^3] - 0 = \tfrac12(-1)+\tfrac14(0)+\tfrac14(8) = -\tfrac12+2 = \tfrac32, and indeed 3.75+2(1.5)=6.753.75 + 2(1.5) = 6.75. ✓

Scope, edge cases & common traps

SituationWhat happensWhy
E[X2]=∞E[X^2] = \inftyVariance is undefined (or +∞+\infty by convention)Second moment must be finite for μ\mu, E[X2]E[X^2] to make the shortcut formula meaningful
Cauchy distributionVariance does not existE[X]E[X] itself fails to exist (integral not absolutely convergent), so variance is undefined, not merely infinite
Cov(X,Y)=0\mathrm{Cov}(X,Y)=0 but X,YX,Y dependentAdditivity of variance can still coincidentally hold or failZero covariance is weaker than independence; see counterexample
nn pairwise independent variablesVariance still addsCorollary only needs pairwise independence, not full mutual independence
Constant random variable X=cX=cVar(X)=0\mathrm{Var}(X) = 0Degenerate case of nonnegativity theorem, equality condition
Negative "variance" from arithmeticSignals a computational errorVariance is a sum/integral of squares — always ≥0\ge 0

The mathematician behind it

Origins · 1 minPafnuty Lvovich Chebyshev and how this was discovered
PL
Pafnuty Lvovich Chebyshev
1821–1894 · Imperial Russia, St. Petersburg mathematical school

Chebyshev was a towering figure of 19th-century Russian mathematics, working across number theory, approximation theory, and probability. His inequality — proved in the 1867 paper Des valeurs moyennes — gave the first fully general, distribution-free bound on how far a random variable can deviate from its mean, using only the existence of a finite variance. It transformed variance from a descriptive statistic into a rigorous tool for proving limit theorems, most famously an early general form of the Law of Large Numbers via variance bounds. Chebyshev founded an influential school (students included Andrei Markov, after whom Markov's inequality is named) that shaped Russian probability for generations.

History
- 1821 — Pafnuty Chebyshev born in Okatovo, Russia.
- 1853 — Chebyshev begins publishing foundational work on approximation and probability theory at St. Petersburg University.
- 1867 — Chebyshev proves the inequality bearing his name, in the memoir Des valeurs moyennes, generalizing earlier special cases due to Bienaymé.
- 1894 — Chebyshev dies in St. Petersburg, leaving a mathematical school that shapes 20th-century probability.
- 1900s — Variance and Chebyshev-type bounds become standard tools underlying the rigorous development of the Central Limit Theorem and statistical estimation theory.
Why do we square deviations instead of taking absolute values?

Both E[(X−μ)2]E[(X-\mu)^2] and E[∣X−μ∣]E[|X-\mu|] (the "mean absolute deviation") are legitimate measures of spread, but squaring has major structural advantages. It makes the function differentiable everywhere (absolute value has a kink at 00), which matters for optimization — the mean μ\mu is exactly the constant cc minimizing E[(X−c)2]E[(X-c)^2], a clean fact with no analogue as clean for absolute deviation. Squaring also produces the beautiful additivity law Var(X+Y)=Var(X)+Var(Y)\mathrm{Var}(X+Y)=\mathrm{Var}(X)+\mathrm{Var}(Y) under independence, because cross terms vanish in expectation; no such simple identity holds for mean absolute deviation. Finally, variance connects directly to the geometry of L2L^2 (Hilbert) spaces, enabling projection and orthogonality arguments used throughout statistics.

Is standard deviation always more meaningful than variance?

For interpretation, generally yes: standard deviation is in the same units as XX, so you can compare σ\sigma directly to μ\mu or to a tolerance threshold (e.g. "the machine's output is 50±250 \pm 2 grams" makes sense with σ\sigma, not σ2\sigma^2). However, variance is usually the more convenient quantity for algebra: the additivity theorem, the bias-variance decomposition, and covariance matrices are all naturally stated in terms of variance because it is linear in the underlying squared-deviation expectations, whereas taking a square root does not commute with addition.

Does a distribution always have a finite variance?

No. A classic counterexample is the Cauchy distribution, whose density decays only like 1/x21/x^2 for large ∣x∣|x| — too slowly for even the first moment E[∣X∣]E[|X|] to converge absolutely, so neither the mean nor the variance exists. Even distributions with a finite mean can have infinite variance, such as certain heavy-tailed Pareto distributions with shape parameter between 11 and 22. In such cases, tools like Chebyshev's inequality simply do not apply, and alternative descriptors (median, interquartile range) are used instead.

What exactly does Chebyshev's inequality guarantee, and how tight is it?

It guarantees that for any random variable with finite variance σ2\sigma^2, the probability of straying kk standard deviations from the mean is at most 1/k21/k^2 — for example, at most 25%25\% chance of being 22 or more standard deviations away, and at most ≈11.1%\approx 11.1\% chance of being 33 or more away. This bound holds for every distribution with finite variance, which is its strength, but it is usually very loose for well-behaved distributions (e.g. the true probability for a normal distribution at k=2k=2 is about 4.55%4.55\%, far below Chebyshev's 25%25\% bound). Its value lies in universality, not sharpness.

How does variance relate to skewness and kurtosis mentioned in the definition?

Variance (second central moment) measures spread but says nothing about shape asymmetry or tail weight. Skewness (third standardized central moment) measures whether the distribution leans left or right: positive skewness means a longer right tail. Kurtosis (fourth standardized central moment, often reported as excess kurtosis relative to the normal's value of 33) measures how heavy the tails are relative to a normal distribution, controlling how often extreme outliers occur. Together, mean, variance, skewness, and kurtosis form the first four "moments" describing location, spread, asymmetry, and tail behavior — a progressively finer description of a distribution's shape.

Why does the proof of Chebyshev's inequality route through Markov's inequality?

Markov's inequality is the more primitive, more general fact: it requires only nonnegativity and a finite mean of the variable in question, with no reference to variance at all. Applying it to the specific nonnegative random variable (X−μ)2(X-\mu)^2 — whose expectation is by definition the variance — instantly produces Chebyshev's bound as a special case. This "reduce to Markov" technique is a standard and reusable strategy: many concentration inequalities (Chernoff bounds, for instance) are proved by applying Markov's inequality to a cleverly chosen transformed random variable, such as etXe^{tX}.

Can two different distributions have the same mean and variance but look completely different?

Yes, easily — mean and variance are only two numbers and cannot pin down an entire distribution. A symmetric bimodal distribution and a unimodal bell-shaped distribution can share identical mean and variance while having very different skewness and kurtosis, or even identical first four moments while differing in the fifth. This is precisely why the moment hierarchy continues indefinitely, and why, in the limit, one considers the full moment generating function or characteristic function to (under suitable conditions) uniquely pin down a distribution — the natural next topic after moments themselves.

Before this

Where this leads