Covariance and Correlation

Suppose you measure the height and weight of many adults. Or the temperature and ice-cream sales on many days. You suspect these pairs of numbers move together. But "moving together" is a vague idea until it is made precise. Covariance and correlation are the two standard tools that quantify this tendency. They measure how two random variables increase or decrease together. This turns an intuitive notion of association into a computable, provably well-behaved number. The idea matters because almost every question in applied probability and statistics involving more than one random variable reduces at some point to computing or bounding a covariance. This includes portfolio risk, regression, principal component analysis, and the variance of a sum of dependent random variables. Covariance is the natural "cross term" that appears whenever we expand Var(X+Y)\mathrm{Var}(X+Y). Correlation is its normalized, unit-free version. It is bounded between −1-1 and 11, which allows comparison across different scales of measurement. The big picture is that covariance is a bilinear form on random variables. It is much like an inner product on a vector space. This algebraic structure is exactly what makes it tractable. It obeys clean symmetry and linearity rules. These rules let us compute variances of complicated linear combinations from simple pairwise pieces. Correlation is the corresponding "cosine of the angle" between two random variables in this inner-product geometry. The Cauchy–Schwarz inequality underlies its boundedness. This article gives the rigorous definitions of covariance and correlation. It proves bilinearity and the Cauchy–Schwarz-based bound ∣ρ∣≤1|\rho| \le 1. It proves that independence implies zero covariance. It also carefully establishes, with an explicit counterexample, that the converse fails. The article works through two fully computed non-trivial examples. It closes by connecting to the variance of sums, the bivariate normal law, and conditional expectation.

Key takeaways
  • Covariance Cov(X,Y)=E[(X−EX)(Y−EY)]\mathrm{Cov}(X,Y) = E[(X-EX)(Y-EY)] measures the linear co-movement of XX and YY, in the units of XX times units of YY.
  • Correlation ρ(X,Y)=Cov(X,Y)/(σXσY)\rho(X,Y) = \mathrm{Cov}(X,Y)/(\sigma_X \sigma_Y) is the unit-free, scale-invariant version, always in [−1,1][-1,1].
  • Covariance is symmetric and bilinear: Cov(aX+b,cY+d)=ac Cov(X,Y)\mathrm{Cov}(aX+b, cY+d) = ac\,\mathrm{Cov}(X,Y), and it distributes over sums.
  • Independence forces Cov(X,Y)=0\mathrm{Cov}(X,Y)=0, but zero covariance does not imply independence — only for jointly Gaussian pairs (with finite variance) does the converse hold.
  • ∣ρ(X,Y)∣=1|\rho(X,Y)| = 1 if and only if YY is an exact affine (linear) function of XX almost surely.
  • Key idea: covariance is a bilinear "inner product" on centered random variables, and correlation is the cosine of the angle it induces — capturing only linear dependence, not dependence in general.

Where this lives today

Covariance and correlation are foundational far beyond introductory probability. In finance, the covariance matrix of asset returns drives Markowitz portfolio optimization, where minimizing portfolio variance for a target return requires exactly the bilinear expansion of Variance of Sums across many assets. In statistics, the correlation coefficient underlies simple linear regression, where the least-squares slope is β^=Cov(X,Y)/Var(X)\hat\beta = \mathrm{Cov}(X,Y)/\mathrm{Var}(X), and its square, ρ2\rho^2, is the coefficient of determination reported in every regression output. In machine learning and signal processing, covariance matrices are diagonalized via principal component analysis to find directions of maximal variance in high-dimensional data, and whitening/decorrelation transforms rely on the bilinear algebra proved above. In physics and engineering, covariance appears in error propagation formulas and in the study of correlated noise processes; in cryptography and coding theory, correlation attacks exploit statistical dependence between key bits and outputs, a direct if adversarial use of the same coefficient. The natural next steps are to see how covariance structures the The Bivariate Normal Distribution (where uncorrelated truly means independent), how Conditional Expectation refines the "explained variance" idea underlying correlation, and how bilinearity generalizes to the multivariate covariance matrix used throughout multivariate statistics.

:::faq[Why does covariance use E[(X−μX)(Y−μY)]E[(X-\mu_X)(Y-\mu_Y)] instead of just E[XY]E[XY]?] Centering removes the effect of the means so covariance measures co-variation around the average, not the raw size of XX and YY. If XX and YY were both shifted upward by adding large constants, E[XY]E[XY] would change substantially even though the underlying association pattern is unchanged, since constants don't create or destroy dependence. The centered version Cov(X,Y)=E[XY]−μXμY\mathrm{Cov}(X,Y) = E[XY]-\mu_X\mu_Y is invariant under such shifts (this is exactly the affine-shift property proved in the Theorem), which is what makes it a meaningful measure of joint variability rather than an artifact of location.

Ask the tutor: How does covariance generalize to a covariance matrix for random vectors, and why must that matrix always be positive semi-definite?

DefinitionCovariance and Correlation

Let XX and YY be random variables on a common probability space (Ω,F,P)(\Omega, \mathcal F, P), each with finite second moment, i.e. E[X2]<∞E[X^2] < \infty and E[Y2]<∞E[Y^2] < \infty. Write μX=E[X]\mu_X = E[X], μY=E[Y]\mu_Y = E[Y]. The covariance of XX and YY is

Cov(X,Y)  =  E[(X−μX)(Y−μY)].\mathrm{Cov}(X,Y) \;=\; E\big[(X-\mu_X)(Y-\mu_Y)\big].

This quantity is well-defined and finite: by the Cauchy–Schwarz inequality for the inner product E[UV]E[UV] on square-integrable random variables,

E[∣X−μX∣∣Y−μY∣]≤E[(X−μX)2] E[(Y−μY)2]=σXσY<∞,E\big[|X-\mu_X||Y-\mu_Y|\big] \le \sqrt{E[(X-\mu_X)^2]}\,\sqrt{E[(Y-\mu_Y)^2]} = \sigma_X \sigma_Y < \infty,

so the expectation defining Cov(X,Y)\mathrm{Cov}(X,Y) converges absolutely. An equivalent and often more convenient computational formula is

Cov(X,Y)=E[XY]−μXμY,\mathrm{Cov}(X,Y) = E[XY] - \mu_X \mu_Y,

obtained by expanding the product (X−μX)(Y−μY)(X-\mu_X)(Y-\mu_Y) and using linearity of expectation. If 0<σX,σY<∞0 < \sigma_X, \sigma_Y < \infty (i.e. XX and YY are each non-degenerate, with strictly positive variance), the correlation coefficient of XX and YY is

ρ(X,Y)  =  Cov(X,Y)σXσY.\rho(X,Y) \;=\; \frac{\mathrm{Cov}(X,Y)}{\sigma_X \sigma_Y}.

If σX=0\sigma_X = 0 or σY=0\sigma_Y = 0 (one variable is almost surely constant), ρ(X,Y)\rho(X,Y) is left undefined, since the variable carries no linear variability to correlate. XX and YY are called uncorrelated if Cov(X,Y)=0\mathrm{Cov}(X,Y) = 0, equivalently ρ(X,Y)=0\rho(X,Y)=0 (when defined).

Intuition

Think of a centered random variable X−μXX - \mu_X as a "vector" in the space of square-integrable random variables, with inner product ⟨U,V⟩=E[UV]\langle U, V\rangle = E[UV]. The covariance Cov(X,Y)\mathrm{Cov}(X,Y) is exactly the inner product of the centered variables X−μXX-\mu_X and Y−μYY-\mu_Y. The variance Var(X)=Cov(X,X)\mathrm{Var}(X) = \mathrm{Cov}(X,X) is the squared length of X−μXX - \mu_X in this geometry. Correlation is then the cosine of the angle between the two centered vectors:

ρ(X,Y)=cos⁡θ=⟨X−μX, Y−μY⟩∥X−μX∥ ∥Y−μY∥.\rho(X,Y) = \cos\theta = \frac{\langle X-\mu_X,\, Y-\mu_Y\rangle}{\|X-\mu_X\|\,\|Y-\mu_Y\|}.

A cosine is always in [−1,1][-1,1]; a value near +1+1 means the vectors point in nearly the same direction (strong positive linear association), near −1-1 means nearly opposite directions, and near 00 means the vectors are nearly orthogonal — no linear relationship, though a strong nonlinear one may still be hiding. This geometric picture also explains why covariance is bilinear: inner products are linear in each argument, so covariance inherits linearity in XX and in YY separately, which is what makes it algebraically tractable when we expand variances of sums. It is crucial to keep in mind that this is a picture of linear association only. Two variables can be perfectly, deterministically dependent — for instance Y=X2Y = X^2 with XX symmetric about 00 — and still have zero covariance, because the "angle" geometry only sees linear alignment, not general functional dependence.

Figure
Scatter clouds and their correlation "angle": three cases with the same variances but different linear association
ρ ≈ 0.9 (strong +) ρ ≈ 0 (no linear trend) ρ ≈ −0.85 (strong −)
TheoremBasic Properties of Covariance and Correlation

Let X,Y,ZX, Y, Z be random variables with finite second moments on (Ω,F,P)(\Omega,\mathcal F, P), and let a,b,c,d∈Ra,b,c,d \in \mathbb{R}. Then: 1. Symmetry: Cov(X,Y)=Cov(Y,X)\mathrm{Cov}(X,Y) = \mathrm{Cov}(Y,X). 2. Relation to variance: Cov(X,X)=Var(X)\mathrm{Cov}(X,X) = \mathrm{Var}(X). 3. Bilinearity:

Cov(aX+bY,Z)=a Cov(X,Z)+b Cov(Y,Z),\mathrm{Cov}(aX+bY, Z) = a\,\mathrm{Cov}(X,Z) + b\,\mathrm{Cov}(Y,Z),

and by symmetry the analogous identity holds in the second argument. 4. Affine shifts: Cov(aX+b, cY+d)=ac Cov(X,Y)\mathrm{Cov}(aX+b,\, cY+d) = ac\,\mathrm{Cov}(X,Y). 5. Variance of a sum:

Var(X+Y)=Var(X)+Var(Y)+2 Cov(X,Y).\mathrm{Var}(X+Y) = \mathrm{Var}(X) + \mathrm{Var}(Y) + 2\,\mathrm{Cov}(X,Y).

6. Independence implies uncorrelatedness: if XX and YY are independent, then Cov(X,Y)=0\mathrm{Cov}(X,Y) = 0. 7. Cauchy–Schwarz bound: if σX,σY∈(0,∞)\sigma_X, \sigma_Y \in (0,\infty), then

∣Cov(X,Y)∣≤σXσY,hence ∣ρ(X,Y)∣≤1.|\mathrm{Cov}(X,Y)| \le \sigma_X \sigma_Y, \qquad\text{hence } |\rho(X,Y)| \le 1.

8. Equality case: ∣ρ(X,Y)∣=1|\rho(X,Y)| = 1 if and only if there exist constants c≠0c \neq 0 and dd such that Y=cX+dY = cX + d almost surely; moreover ρ(X,Y)=1\rho(X,Y)=1 corresponds to c>0c>0 and ρ(X,Y)=−1\rho(X,Y)=-1 to c<0c<0.

Proof

We prove each part in turn, using only linearity of expectation and previously established parts.

(1) Symmetry. By definition, Cov(X,Y)=E[(X−μX)(Y−μY)]\mathrm{Cov}(X,Y) = E[(X-\mu_X)(Y-\mu_Y)]. Since multiplication of real numbers is commutative, (X−μX)(Y−μY)=(Y−μY)(X−μX)(X-\mu_X)(Y-\mu_Y) = (Y-\mu_Y)(X-\mu_X) pointwise on Ω\Omega, so the two expectations coincide.

(2) Relation to variance. Set Y=XY=X: Cov(X,X)=E[(X−μX)2]=Var(X)\mathrm{Cov}(X,X) = E[(X-\mu_X)^2] = \mathrm{Var}(X), the standard definition of variance.

(3) Bilinearity. Let W=aX+bYW = aX+bY; then μW=aμX+bμY\mu_W = a\mu_X + b\mu_Y by linearity of expectation, and pointwise

W−μW=a(X−μX)+b(Y−μY).W - \mu_W = a(X-\mu_X) + b(Y-\mu_Y).

Hence

Cov(W,Z)=E[(W−μW)(Z−μZ)]=E[a(X−μX)(Z−μZ)+b(Y−μY)(Z−μZ)].\mathrm{Cov}(W,Z) = E\big[(W-\mu_W)(Z-\mu_Z)\big] = E\big[a(X-\mu_X)(Z-\mu_Z) + b(Y-\mu_Y)(Z-\mu_Z)\big].

By linearity of expectation this equals

a E[(X−μX)(Z−μZ)]+b E[(Y−μY)(Z−μZ)]=a Cov(X,Z)+b Cov(Y,Z).a\,E[(X-\mu_X)(Z-\mu_Z)] + b\,E[(Y-\mu_Y)(Z-\mu_Z)] = a\,\mathrm{Cov}(X,Z) + b\,\mathrm{Cov}(Y,Z).

All expectations above are finite by the Cauchy–Schwarz argument in the Definition applied to each pair, so no convergence issues arise, and linearity of expectation for two finite-mean random variables is legitimate to invoke.

(4) Affine shifts. Apply bilinearity in the first slot with the constant bb absorbed: since a constant bb has Cov(b,Z)=E[(b−b)(Z−μZ)]=0\mathrm{Cov}(b, Z) = E[(b-b)(Z-\mu_Z)] = 0 for any ZZ (as bb is deterministic, its "centered" version is identically 00), bilinearity (part 3, applied twice, once in each argument) gives

Cov(aX+b,cY+d)=a Cov(X,cY+d)+Cov(b,cY+d)=a[c Cov(X,Y)+Cov(X,d)]+0=ac Cov(X,Y),\mathrm{Cov}(aX+b, cY+d) = a\,\mathrm{Cov}(X, cY+d) + \mathrm{Cov}(b, cY+d) = a\big[c\,\mathrm{Cov}(X,Y) + \mathrm{Cov}(X,d)\big] + 0 = ac\,\mathrm{Cov}(X,Y),

using Cov(X,d)=0\mathrm{Cov}(X,d) = 0 by the same constant argument applied in the second slot.

(5) Variance of a sum. By part (2) and bilinearity applied twice (first in the outer argument, then distributing over X+YX+Y in the inner argument),

Var(X+Y)=Cov(X+Y,X+Y)=Cov(X,X)+Cov(X,Y)+Cov(Y,X)+Cov(Y,Y).\mathrm{Var}(X+Y) = \mathrm{Cov}(X+Y, X+Y) = \mathrm{Cov}(X,X) + \mathrm{Cov}(X,Y) + \mathrm{Cov}(Y,X) + \mathrm{Cov}(Y,Y).

By symmetry (part 1), Cov(Y,X)=Cov(X,Y)\mathrm{Cov}(Y,X) = \mathrm{Cov}(X,Y), and by part (2), Cov(X,X)=Var(X)\mathrm{Cov}(X,X)=\mathrm{Var}(X), Cov(Y,Y)=Var(Y)\mathrm{Cov}(Y,Y) = \mathrm{Var}(Y). Substituting gives the claimed identity.

(6) Independence implies uncorrelatedness. If X⊥YX \perp Y, then for any Borel-measurable functions g,hg,h with E∣g(X)∣,E∣h(Y)∣<∞E|g(X)|, E|h(Y)| <\infty and E∣g(X)h(Y)∣<∞E|g(X)h(Y)| < \infty, independence gives the factorization E[g(X)h(Y)]=E[g(X)]E[h(Y)]E[g(X)h(Y)] = E[g(X)]E[h(Y)] (this is the standard product rule for expectations of independent random variables, established from the definition of independence and Fubini's theorem for the joint law PX⊗PYP_X \otimes P_Y). Taking g(x)=xg(x) = x, h(y)=yh(y) = y (valid here since E[X2],E[Y2]<∞E[X^2], E[Y^2] <\infty implies E∣XY∣≤E[X2]E[Y2]<∞E|XY| \le \sqrt{E[X^2]E[Y^2]} <\infty by Cauchy–Schwarz), we get

E[XY]=E[X]E[Y]=μXμY.E[XY] = E[X]E[Y] = \mu_X \mu_Y.

Hence, by the computational formula in the Definition,

Cov(X,Y)=E[XY]−μXμY=μXμY−μXμY=0.\mathrm{Cov}(X,Y) = E[XY] - \mu_X\mu_Y = \mu_X\mu_Y - \mu_X\mu_Y = 0.

(7) Cauchy–Schwarz bound. Let U=X−μXU = X - \mu_X, V=Y−μYV = Y-\mu_Y; these are square-integrable centered random variables. The Cauchy–Schwarz inequality for the bilinear form ⟨U,V⟩:=E[UV]\langle U,V\rangle := E[UV] states

∣E[UV]∣≤E[U2]E[V2].|E[UV]| \le \sqrt{E[U^2]}\sqrt{E[V^2]}.

(Proof of this inequality: for every real tt, 0≤E[(U+tV)2]=E[U2]+2tE[UV]+t2E[V2]0 \le E[(U+tV)^2] = E[U^2] + 2tE[UV] + t^2 E[V^2], a quadratic in tt that is nonnegative for all tt; a real quadratic At2+Bt+C≥0At^2+Bt+C \ge 0 for all tt requires discriminant B2−4AC≤0B^2 - 4AC \le 0, i.e. 4E[UV]2≤4E[U2]E[V2]4E[UV]^2 \le 4E[U^2]E[V^2], giving the stated bound. If E[V2]=0E[V^2] = 0 the inequality is immediate since then V=0V=0 a.s. and both sides vanish.) Applying this with U,VU,V as above, and noting E[U2]=σX2E[U^2] = \sigma_X^2, E[V2]=σY2E[V^2] = \sigma_Y^2:

∣Cov(X,Y)∣=∣E[UV]∣≤σXσY.|\mathrm{Cov}(X,Y)| = |E[UV]| \le \sigma_X \sigma_Y.

Dividing both sides by σXσY>0\sigma_X\sigma_Y > 0 gives ∣ρ(X,Y)∣≤1|\rho(X,Y)| \le 1.

(8) Equality case. From the proof of Cauchy–Schwarz above, equality ∣E[UV]∣=E[U2]E[V2]|E[UV]| = \sqrt{E[U^2]E[V^2]} holds if and only if the discriminant is exactly zero, which happens if and only if the quadratic t↦E[(U+tV)2]t \mapsto E[(U+tV)^2] attains the value 00 at its minimizing t=t0=−E[UV]/E[V2]t = t_0 = -E[UV]/E[V^2]. But E[(U+t0V)2]=0E[(U+t_0V)^2] = 0 for a random variable means U+t0V=0U + t_0 V = 0 almost surely (a nonnegative-expectation random variable, namely (U+t0V)2≥0(U+t_0V)^2 \ge 0, has expectation zero iff it is 00 a.s. — this is the standard fact that E[Z]=0E[Z]=0 with Z≥0Z\ge0 forces Z=0Z=0 a.s., proved by noting P(Z>1/n)≤nE[Z]=0P(Z > 1/n) \le n E[Z] = 0 for every nn by Markov's inequality, and taking a union over nn). Thus U=−t0VU = -t_0 V a.s., i.e.

X−μX=−t0(Y−μY)a.s.X - \mu_X = -t_0 (Y - \mu_Y) \quad\text{a.s.}

which rearranges to X=−t0Y+(μX+t0μY)X = -t_0 Y + (\mu_X + t_0\mu_Y) a.s., an exact affine relation; solving for YY in terms of XX (since t0≠0t_0 \ne 0 when ρ≠0\rho \ne 0, which holds here as ∣ρ∣=1≠0|\rho|=1\ne0) gives Y=cX+dY = cX+d a.s. for constants c=−1/t0c = -1/t_0, dd. Conversely, if Y=cX+dY = cX+d a.s. with c≠0c\neq 0, then by part (4), Cov(X,Y)=Cov(X,cX+d)=c Var(X)\mathrm{Cov}(X,Y) = \mathrm{Cov}(X, cX+d) = c\,\mathrm{Var}(X), and σY2=Var(cX+d)=c2Var(X)\sigma_Y^2 = \mathrm{Var}(cX+d) = c^2\mathrm{Var}(X) so σY=∣c∣σX\sigma_Y = |c|\sigma_X; hence

ρ(X,Y)=c Var(X)σX⋅∣c∣σX=c∣c∣=sign(c)∈{+1,−1}.\rho(X,Y) = \frac{c\,\mathrm{Var}(X)}{\sigma_X \cdot |c|\sigma_X} = \frac{c}{|c|} = \mathrm{sign}(c) \in \{+1,-1\}.

This confirms ∣ρ∣=1|\rho|=1 exactly in the affine case, with sign matching the sign of cc, completing the equivalence. ■\blacksquare

∎

Worked examples

Example 1 — A discrete joint distribution with negative, non-trivial correlation. Let (X,Y)(X,Y) take values in {0,1,2}×{1,2}\{0,1,2\} \times \{1,2\} with joint probability mass function given by the table:

X\YX\backslash Y1122
000.100.100.300.30
110.250.250.050.05
220.200.200.100.10

(These sum to 11: 0.10+0.30+0.25+0.05+0.20+0.10=1.000.10+0.30+0.25+0.05+0.20+0.10 = 1.00.) Marginals. P(X=0)=0.40P(X=0) = 0.40, P(X=1)=0.30P(X=1)=0.30, P(X=2)=0.30P(X=2)=0.30. Check: 0.4+0.3+0.3=10.4+0.3+0.3=1. P(Y=1)=0.10+0.25+0.20=0.55P(Y=1) = 0.10+0.25+0.20 = 0.55, P(Y=2)=0.30+0.05+0.10=0.45P(Y=2) = 0.30+0.05+0.10=0.45. Check: 0.55+0.45=10.55+0.45=1. Means.

E[X]=0(0.4)+1(0.3)+2(0.3)=0+0.3+0.6=0.9.E[X] = 0(0.4)+1(0.3)+2(0.3) = 0 + 0.3+0.6 = 0.9.
E[Y]=1(0.55)+2(0.45)=0.55+0.90=1.45.E[Y] = 1(0.55) + 2(0.45) = 0.55+0.90=1.45.

Second moments.

E[X2]=02(0.4)+12(0.3)+22(0.3)=0+0.3+1.2=1.5,E[X^2] = 0^2(0.4) + 1^2(0.3) + 2^2(0.3) = 0 + 0.3 + 1.2 = 1.5,
Var(X)=1.5−0.92=1.5−0.81=0.69.\mathrm{Var}(X) = 1.5 - 0.9^2 = 1.5 - 0.81 = 0.69.
E[Y2]=12(0.55)+22(0.45)=0.55+1.80=2.35,E[Y^2] = 1^2(0.55) + 2^2(0.45) = 0.55 + 1.80 = 2.35,
Var(Y)=2.35−1.452=2.35−2.1025=0.2475.\mathrm{Var}(Y) = 2.35 - 1.45^2 = 2.35 - 2.1025 = 0.2475.

Cross moment. E[XY]=∑x,yxy p(x,y)E[XY] = \sum_{x,y} xy\,p(x,y); only terms with x≠0x\ne0 contribute:

E[XY]=(1)(1)(0.25)+(1)(2)(0.05)+(2)(1)(0.20)+(2)(2)(0.10)E[XY] = (1)(1)(0.25) + (1)(2)(0.05) + (2)(1)(0.20) + (2)(2)(0.10)
=0.25+0.10+0.40+0.40=1.15.= 0.25 + 0.10 + 0.40 + 0.40 = 1.15.

Covariance.

Cov(X,Y)=E[XY]−E[X]E[Y]=1.15−(0.9)(1.45)=1.15−1.305=−0.155.\mathrm{Cov}(X,Y) = E[XY] - E[X]E[Y] = 1.15 - (0.9)(1.45) = 1.15 - 1.305 = -0.155.

Correlation.

σX=0.69≈0.8307,σY=0.2475≈0.4975.\sigma_X = \sqrt{0.69} \approx 0.8307, \qquad \sigma_Y = \sqrt{0.2475}\approx 0.4975.
ρ(X,Y)=−0.1550.8307×0.4975≈−0.1550.41328≈−0.3751.\rho(X,Y) = \frac{-0.155}{0.8307 \times 0.4975} \approx \frac{-0.155}{0.41328} \approx -0.3751.

So XX and YY are negatively correlated: as XX tends to be larger, YY tends to be smaller (probability mass leans toward Y=2Y=2 when X=0X=0, and toward Y=1Y=1 when X≥1X\ge 1). The moderate magnitude ∣ρ∣≈0.375|\rho|\approx 0.375 reflects a real but far-from-deterministic linear association, consistent with ∣ρ∣≤1|\rho| \le 1 from the Theorem proved above.

Example 2 — Covariance of a linear combination via bilinearity, with a non-independent, non-Gaussian pair. Let U∼Uniform{−1,0,1,2}U \sim \mathrm{Uniform}\{-1,0,1,2\} (each value with probability 1/41/4), and let ε\varepsilon be an independent random variable with P(ε=1)=P(ε=−1)=1/2P(\varepsilon = 1) = P(\varepsilon=-1) = 1/2. Define

X=U,Y=U+2Uε=U(1+2ε).X = U, \qquad Y = U + 2U\varepsilon = U(1+2\varepsilon).

Note YY depends on X=UX=U multiplicatively through the noise ε\varepsilon, so X,YX, Y are dependent but not simply affinely related, and ε\varepsilon is not independent of YY's relationship to XX in a simple additive-noise sense — this is a genuinely asymmetric, non-symmetric construction. We want Cov(X,Y)\mathrm{Cov}(X,Y) using bilinearity rather than brute-force joint enumeration. Setup via bilinearity. Write Y=U+2UεY = U + 2U\varepsilon. Then

Cov(X,Y)=Cov(U, U+2Uε)=Cov(U,U)+2 Cov(U,Uε)\mathrm{Cov}(X,Y) = \mathrm{Cov}(U,\, U+2U\varepsilon) = \mathrm{Cov}(U,U) + 2\,\mathrm{Cov}(U, U\varepsilon)

by bilinearity (Theorem, part 3), applied with a=1a=1 for the UU term absorbed into itself. So we need Var(U)\mathrm{Var}(U) and Cov(U,Uε)\mathrm{Cov}(U,U\varepsilon). Moments of UU. Values −1,0,1,2-1,0,1,2 each with probability 1/41/4:

E[U]=14(−1+0+1+2)=24=0.5.E[U] = \tfrac14(-1+0+1+2) = \tfrac{2}{4} = 0.5.
E[U2]=14(1+0+1+4)=64=1.5.E[U^2] = \tfrac14(1+0+1+4) = \tfrac{6}{4} = 1.5.
Var(U)=1.5−0.25=1.25.\mathrm{Var}(U) = 1.5 - 0.25 = 1.25.

Cross term Cov(U,Uε)\mathrm{Cov}(U, U\varepsilon). First compute E[Uε]E[U\varepsilon]. Since U⊥εU \perp \varepsilon and E[ε]=0E[\varepsilon]=0,

E[Uε]=E[U]E[ε]=0.5×0=0.E[U\varepsilon] = E[U]E[\varepsilon] = 0.5 \times 0 = 0.

Next, E[U⋅Uε]=E[U2ε]=E[U2]E[ε]E[U \cdot U\varepsilon] = E[U^2 \varepsilon] = E[U^2]E[\varepsilon] (again since U⊥εU \perp \varepsilon, so U2⊥εU^2 \perp \varepsilon, and functions of independent variables are independent) =1.5×0=0= 1.5 \times 0 = 0. So Cov(U,Uε)=E[U⋅Uε]−E[U]E[Uε]=0−(0.5)(0)=0\mathrm{Cov}(U, U\varepsilon) = E[U\cdot U\varepsilon] - E[U]E[U\varepsilon] = 0 - (0.5)(0) = 0. Assembling.

Cov(X,Y)=Var(U)+2(0)=1.25.\mathrm{Cov}(X,Y) = \mathrm{Var}(U) + 2(0) = 1.25.

Correlation. We also need Var(Y)\mathrm{Var}(Y). Write Y=U(1+2ε)Y = U(1+2\varepsilon); since ε2=1\varepsilon^2=1 always,

Y2=U2(1+2ε)2=U2(1+4ε+4ε2)=U2(5+4ε),Y^2 = U^2(1+2\varepsilon)^2 = U^2(1+4\varepsilon+4\varepsilon^2) = U^2(5+4\varepsilon),

using ε2=1\varepsilon^2=1. So

E[Y2]=E[U2(5+4ε)]=5E[U2]+4E[U2ε]=5(1.5)+4(E[U2]E[ε])=7.5+4(1.5)(0)=7.5.E[Y^2] = E[U^2(5+4\varepsilon)] = 5E[U^2] + 4E[U^2\varepsilon] = 5(1.5) + 4\big(E[U^2]E[\varepsilon]\big) = 7.5 + 4(1.5)(0) = 7.5.

Also E[Y]=E[U]+2E[Uε]=0.5+2(0)=0.5E[Y] = E[U] + 2E[U\varepsilon] = 0.5 + 2(0) = 0.5. So

Var(Y)=E[Y2]−(E[Y])2=7.5−0.25=7.25.\mathrm{Var}(Y) = E[Y^2] - (E[Y])^2 = 7.5 - 0.25 = 7.25.

Then

ρ(X,Y)=Cov(X,Y)Var(X)Var(Y)=1.251.257.25=1.259.0625=1.253.0104≈0.4152.\rho(X,Y) = \frac{\mathrm{Cov}(X,Y)}{\sqrt{\mathrm{Var}(X)}\sqrt{\mathrm{Var}(Y)}} = \frac{1.25}{\sqrt{1.25}\sqrt{7.25}} = \frac{1.25}{\sqrt{9.0625}} = \frac{1.25}{3.0104} \approx 0.4152.

So even though YY is built from UU multiplied by symmetric noise (making the relationship highly nonlinear and far from a clean line), the pair is moderately positively correlated (ρ≈0.415\rho \approx 0.415) — because larger ∣U∣|U| pushes both X=UX=U (in magnitude and, on the positive side, in value) and the variance of YY upward, while the mean structure still leaves a net positive linear alignment. This illustrates bilinearity as a genuine computational tool, reducing a seemingly complicated joint calculation to variance and cross-term pieces.

CounterexampleZero covariance does not imply independence

The Theorem proved above shows independence ⇒\Rightarrow uncorrelatedness (part 6). The converse is false in general; dropping the hypothesis "independent" and replacing it merely with "uncorrelated" breaks the implication that the variables carry no relationship at all.

Construction. Let XX be a random variable with P(X=−1)=P(X=0)=P(X=1)=13P(X=-1) = P(X=0) = P(X=1) = \tfrac13, and define Y=X2Y = X^2. Then YY is a deterministic function of XX — about as dependent as two random variables can be, since knowing XX tells you YY exactly — yet we show Cov(X,Y)=0\mathrm{Cov}(X,Y)=0. Moments.

E[X]=13(−1+0+1)=0.E[X] = \tfrac13(-1+0+1) = 0.
E[Y]=E[X2]=13(1+0+1)=23.E[Y] = E[X^2] = \tfrac13(1+0+1) = \tfrac{2}{3}.
E[XY]=E[X3]=13((−1)3+03+13)=13(−1+0+1)=0.E[XY] = E[X^3] = \tfrac13\big((-1)^3 + 0^3 + 1^3\big) = \tfrac13(-1+0+1) = 0.

Covariance.

Cov(X,Y)=E[XY]−E[X]E[Y]=0−(0)(23)=0.\mathrm{Cov}(X,Y) = E[XY] - E[X]E[Y] = 0 - (0)\left(\tfrac23\right) = 0.

So XX and Y=X2Y=X^2 are uncorrelated. But they are not independent: for instance,

P(X=0, Y=0)=P(X=0)=13,P(X=0)P(Y=0)=13×13=19,P(X=0,\, Y=0) = P(X=0) = \tfrac13, \qquad P(X=0)P(Y=0) = \tfrac13 \times \tfrac13 = \tfrac19,

and 13≠19\tfrac13 \ne \tfrac19, so the independence factorization fails outright — indeed YY is a deterministic (hence maximally dependent, non-invertible) function of XX.

Why this happens. Covariance only detects linear co-movement: it is built from E[XY]E[XY], a single bilinear pairing. The relationship Y=X2Y=X^2 is symmetric under X↦−XX \mapsto -X while the "linear detector" XY=X3XY = X^3 is antisymmetric under the same flip, so the positive and negative correlations exactly cancel in the expectation, even though the dependence (in fact, a deterministic functional link) is total. This is precisely why the theorem's hypothesis "independent" cannot be weakened to "uncorrelated" when concluding a strong dependence statement — uncorrelated is a strictly weaker, purely linear notion. (The converse does hold under an added hypothesis: if (X,Y)(X,Y) is jointly Gaussian, then uncorrelated implies independent — a special extra structural assumption absent here.)

Scope, edge cases & common traps

SituationWhat holdsWhat can fail
X,YX,Y independent, finite variancesCov(X,Y)=0\mathrm{Cov}(X,Y)=0 always—
X,YX,Y uncorrelatedCov(X,Y)=0\mathrm{Cov}(X,Y)=0Independence need not hold (see counterexample)
X,YX,Y jointly Gaussian, uncorrelatedIndependence does hold (special extra theorem)Requires joint normality, not just each marginal Gaussian
Var(X)=0\mathrm{Var}(X)=0 (constant a.s.)Cov(X,Y)=0\mathrm{Cov}(X,Y)=0 for all YYρ(X,Y)\rho(X,Y) is undefined, not 00
Y=cX+dY=cX+d a.s., c≠0c\ne0∣ρ(X,Y)∣=1|\rho(X,Y)|=1 exactlySign of ρ\rho matches sign of cc
Infinite variance (E[X2]=∞E[X^2]=\infty)Covariance may not existCauchy–Schwarz step in Definition breaks down

The mathematician behind it

Origins · 1 minFrancis Galton and how this was discovered
FG
Francis Galton
1822–1911 · Victorian polymath, founder of biometrics and regression analysis

Francis Galton, studying heredity of physical traits such as height, discovered "regression toward mediocrity" — the tendency of offspring measurements to lie closer to the population mean than their parents' measurements. To quantify the strength of association between paired measurements (parent height and child height, for instance), Galton and his collaborator, the mathematician Karl Pearson, developed the correlation coefficient in the form used today. Pearson's 1896 paper gave the coefficient its now-standard formula and statistical treatment, building directly on Galton's graphical and geometric insights about regression lines through scatter data.

History
- 1877 — Galton introduces regression ideas while studying sweet pea seed sizes across generations.
- 1885 — Galton's "Regression Towards Mediocrity in Hereditary Stature" formalizes regression to the mean using height data.
- 1888 — Galton proposes an early index of co-relation for paired biometric measurements.
- 1896 — Karl Pearson publishes the rigorous mathematical treatment of the product-moment correlation coefficient.
- 1900s — Pearson and the biometric school extend covariance/correlation to multivariate and regression theory broadly used today.
Is correlation the same as causation?

No. A nonzero correlation only records a statistical linear association between two random variables observed together; it says nothing about which, if either, causes the other, or whether both are driven by a third confounding variable. Classic examples include ice cream sales and drowning incidents, both driven by hot weather rather than one causing the other. Establishing causation requires additional structure — randomized experiments, causal graphical models, or instrumental variables — well beyond what the covariance/correlation formalism alone can supply.

Why is correlation always between −1-1 and 11?

This follows directly from the Cauchy–Schwarz inequality applied to the centered random variables X−μXX-\mu_X and Y−μYY-\mu_Y, treated as vectors under the inner product E[UV]E[UV]. Cauchy–Schwarz states that the inner product of two vectors is bounded in absolute value by the product of their norms, and dividing covariance by σXσY\sigma_X\sigma_Y is precisely dividing by that product of norms. The proof (given in the Theorem's proof) uses that a real quadratic in tt, namely E[(U+tV)2]≥0E[(U+tV)^2]\ge0 for all tt, must have non-positive discriminant.

Can covariance be negative, and what does that mean?

Yes; negative covariance means that when XX tends to be above its mean, YY tends to be below its mean, and vice versa — an inverse linear tendency. For example, in Worked Example 1 above, Cov(X,Y)≈−0.155\mathrm{Cov}(X,Y) \approx -0.155 reflects that larger values of XX shift probability mass toward the smaller value of YY. The sign of covariance (and correlation) records the direction of linear association, while the magnitude (especially after normalizing to correlation) records its strength.

Does zero correlation mean the variables carry no information about each other?

Not necessarily — it only means there's no linear relationship. The counterexample Y=X2Y=X^2 with symmetric XX shows a case of exact deterministic dependence with zero correlation, because the dependence is purely nonlinear (quadratic) and the linear detector E[XY]E[XY] cannot see it. To capture more general dependence, statisticians use tools like mutual information, distance correlation, or rank correlations (Spearman's, Kendall's), which are sensitive to monotonic or general dependence rather than only linear dependence.

How does correlation relate to the slope of a regression line?

In simple linear regression of YY on XX, minimizing E[(Y−a−bX)2]E[(Y-a-bX)^2] over a,ba,b yields the optimal slope b∗=Cov(X,Y)/Var(X)b^* = \mathrm{Cov}(X,Y)/\mathrm{Var}(X), which can be rewritten as b∗=ρ(X,Y) σY/σXb^* = \rho(X,Y)\,\sigma_Y/\sigma_X. So correlation is the slope rescaled to be unit-free: it tells you the strength of the best linear fit independent of the measurement units of XX and YY, while the regression slope itself carries the units of YY per unit of XX.

Why does the theorem require finite variance, and what if variances are infinite?

Finite variance guarantees, via Cauchy–Schwarz, that E[∣X−μX∣∣Y−μY∣]<∞E[|X-\mu_X||Y-\mu_Y|] < \infty, so the defining expectation for covariance converges absolutely and the quantity is well-defined. If E[X2]=∞E[X^2]=\infty (as for certain heavy-tailed distributions such as some Pareto or Cauchy-type laws), the covariance may fail to exist altogether, and the entire Cauchy–Schwarz argument underlying ∣ρ∣≤1|\rho|\le1 collapses since one side of the inequality is infinite. In such heavy-tailed settings, robust alternatives like rank-based correlations are typically used instead.

Before this

Where this leads