Independence of Random Variables

Suppose you flip a fair coin and roll a fair die at the same time. Does knowing the coin landed heads tell you anything about the die? Common sense says no — the two physical processes do not interact. Probability theory makes this intuition precise through the notion of independence. Independence is arguably the single most important simplifying assumption in probability. It is what lets us multiply probabilities, add variances, and prove laws of large numbers and central limit theorems. Without it, joint distributions can encode arbitrarily tangled dependencies. Such dependencies resist any general analysis. The idea matters because most of probability's computational machinery relies on it. This includes the binomial distribution, Monte Carlo simulation, and the statistical theory of random sampling. All are built on sequences of independent random variables. Independence is not merely "uncorrelated" or "the events don't seem related." It is a strong, exactly quantifiable condition on the joint distribution. Confusing independence with the weaker condition of zero correlation is a common error. This mistake is frequent in applied probability and statistics. The big picture is that independence admits several logically equivalent formulations. These include factorization of the joint CDF, and factorization of the joint density or mass function. They also include factorization of expectations of products of functions. For events, there is the multiplication rule. Each formulation is useful in different settings. Factorized densities are convenient for computing integrals. The expectation-factorization criterion is the cleanest way to prove independence is preserved under transformations. This article gives the rigorous definition of independence for two and for many random variables. It proves the equivalence of its main characterizations. It works through fully asymmetric numerical examples. It shows precisely what fails when independence is weakened. This weakening can be to pairwise independence or to zero correlation. It sets up the machinery needed for Sums of Random Variables and Convolution, Covariance and Correlation, and The Weak Law of Large Numbers.

Key takeaways
  • Independence of XX and YY means their joint CDF factors: FX,Y(x,y)=FX(x)FY(y)F_{X,Y}(x,y) = F_X(x)F_Y(y) for all x,yx,y.
  • Equivalently, joint density/mass factors as fX,Y(x,y)=fX(x)fY(y)f_{X,Y}(x,y) = f_X(x) f_Y(y) wherever it is defined.
  • Equivalently again, E[g(X)h(Y)]=E[g(X)]E[h(Y)]E[g(X)h(Y)] = E[g(X)]E[h(Y)] for all bounded measurable g,hg,h.
  • Independence implies zero covariance, but the converse is false in general.
  • Pairwise independence of a collection does not imply mutual (joint) independence.
  • Key idea: independence is a factorization statement about the entire joint law, not just about a summary statistic like correlation.

Where this lives today

Independence assumptions drive statistical inference: simple random sampling assumes independent, identically distributed (i.i.d.) observations, which justifies standard error formulas and hypothesis tests. In machine learning, the "naive Bayes" classifier explicitly assumes conditional independence of features given the class label, trading some accuracy for tractable computation. Cryptography relies on independence of key bits and of successive outputs of pseudo-random generators; a detectable dependence between output bits is treated as a cryptographic weakness. In statistical physics, independence (or its controlled failure, as in Markov chains and correlated spin systems) underlies mean-field approximations. In queueing theory and network engineering, independent interarrival and service times make Poisson process models and queueing formulas tractable. Independence is also the structural hypothesis behind Sums of Random Variables and Convolution (where independence lets you compute the distribution of X+YX+Y by convolving densities), behind Covariance and Correlation (which measures how far a joint distribution departs from independence, at least linearly), and behind The Weak Law of Large Numbers (which requires independence, or a suitable weakening, to guarantee sample averages concentrate around the mean).

:::quiz[Which technique is used in the theorem's proof to extend the CDF factorization from rays (−∞,x](-\infty,x] to all Borel sets?]

  • Integration by parts
  • The Cauchy–Schwarz inequality
  • Dynkin's π\pi-λ\lambda theorem applied to a generating π\pi-system ✓
  • Direct computation for every possible Borel set one at a time

:::faq[If XX and YY are independent, is E[XY]E[XY] always equal to E[X]E[Y]E[X]E[Y], even if X,YX,Y are not integrable?] The factorization E[XY]=E[X]E[Y]E[XY]=E[X]E[Y] requires integrability care: if XX or YY fails to have finite expectation, both sides may be undefined or the identity may fail to make sense (e.g. ∞−∞\infty - \infty situations). The theorem's expectation-factorization characterization is stated cleanly for bounded measurable g,hg,h, and extended to nonnegative g,hg,h via monotone convergence, and to general integrable g,hg,h provided the relevant expectations exist; independence guarantees that if X,YX,Y are independent and both integrable, then XYXY is integrable and E[XY]=E[X]E[Y]E[XY]=E[X]E[Y] holds without further conditions.

Ask the tutor: Why does independence of jointly Gaussian, uncorrelated random variables hold even though it fails in general?

DefinitionIndependence of Random Variables

Let XX and YY be random variables defined on a common probability space (Ω,F,P)(\Omega, \mathcal{F}, P), with joint cumulative distribution function

FX,Y(x,y)=P(X≤x, Y≤y),x,y∈R.F_{X,Y}(x,y) = P(X \le x,\, Y \le y), \qquad x, y \in \mathbb{R}.

Let FX(x)=P(X≤x)F_X(x) = P(X \le x) and FY(y)=P(Y≤y)F_Y(y) = P(Y \le y) denote the marginal CDFs. We say XX and YY are independent if

FX,Y(x,y)=FX(x) FY(y)for all x,y∈R.F_{X,Y}(x,y) = F_X(x)\, F_Y(y) \quad \text{for all } x, y \in \mathbb{R}.

More generally, random variables X1,…,XnX_1, \dots, X_n are mutually independent (or simply independent) if their joint CDF factors:

FX1,…,Xn(x1,…,xn)=∏i=1nFXi(xi)for all (x1,…,xn)∈Rn.F_{X_1,\dots,X_n}(x_1,\dots,x_n) = \prod_{i=1}^n F_{X_i}(x_i) \quad \text{for all } (x_1,\dots,x_n) \in \mathbb{R}^n.

An infinite collection {Xi}i∈I\{X_i\}_{i \in I} is independent if every finite subcollection is independent in the above sense. Two events A,B∈FA, B \in \mathcal{F} are independent if P(A∩B)=P(A)P(B)P(A \cap B) = P(A)P(B); this is recovered as the special case X=1AX = \mathbf{1}_A, Y=1BY = \mathbf{1}_B.

Intuition

Think of the joint CDF FX,YF_{X,Y} as encoding all information about how XX and YY relate — not just their individual behavior, but how knowledge of one constrains the other. Independence says this joint object contains no more information than the two marginals separately: you can reconstruct the whole joint law by multiplying the two one-dimensional pieces. Geometrically, when X,YX,Y have a joint density, independence forces the density's level sets and support to behave like a product — probability mass spreads over a rectangle-like grid whose "row totals" and "column totals" alone determine every cell. A helpful mental picture: partition the plane into a grid of thin vertical strips {X∈dx}\{X \in dx\} and horizontal strips {Y∈dy}\{Y \in dy\}. Under independence, the probability assigned to the small rectangle dx×dydx \times dy is exactly the product of the strip probabilities — no cell is "boosted" or "suppressed" relative to what the marginals alone predict. This is why independence is so much stronger than "no linear trend" (zero correlation): correlation only checks a linear summary, while independence checks every cell of the grid simultaneously, including nonlinear dependencies like Y=X2Y = X^2 hiding inside a symmetric marginal.

Figure
Joint mass table: independent vs. dependent joint distributions with identical marginals
Independent joint (product of marginals) Dependent joint (same marginals) X=0 X=1 X=2 .12 .18 .30 .08 .12 .20 Y=0 Y=1 P(X=1,Y=0)=.18=P(X=1)P(Y=0)=.30×.60 X=0 X=1 X=2 .20 .00 .40 .00 .30 .10 Same row/col totals, but P(X=1,Y=0)=0≠.18

The main theorem

TheoremEquivalent Characterizations of Independence

Let XX and YY be random variables on a probability space (Ω,F,P)(\Omega,\mathcal{F},P). The following statements are equivalent.

(a) (CDF factorization) FX,Y(x,y)=FX(x)FY(y)F_{X,Y}(x,y) = F_X(x) F_Y(y) for all x,y∈Rx,y \in \mathbb{R}.

(b) (Event factorization) For all Borel sets A,B⊆RA, B \subseteq \mathbb{R},

P(X∈A, Y∈B)=P(X∈A) P(Y∈B).P(X \in A,\, Y \in B) = P(X \in A)\, P(Y \in B).

(c) (Expectation factorization) For all bounded Borel-measurable g,h:R→Rg, h: \mathbb{R} \to \mathbb{R},

E[g(X)h(Y)]=E[g(X)] E[h(Y)],E[g(X) h(Y)] = E[g(X)]\, E[h(Y)],

and both sides are well defined whenever g(X),h(Y)g(X), h(Y) are integrable (in particular this includes the case g,hg,h merely nonnegative and Borel measurable, by monotone convergence).

(d) (Density/mass factorization, when applicable) If X,YX, Y are both discrete, (a) holds if and only if

pX,Y(x,y)=pX(x) pY(y)for all x,y.p_{X,Y}(x,y) = p_X(x)\, p_Y(y) \quad \text{for all } x, y.

If (X,Y)(X,Y) has a joint density fX,Yf_{X,Y}, then (a) holds if and only if XX and YY each have densities fX,fYf_X, f_Y with

fX,Y(x,y)=fX(x) fY(y)for (Lebesgue-)almost every (x,y).f_{X,Y}(x,y) = f_X(x)\, f_Y(y) \quad \text{for (Lebesgue-)almost every } (x,y).
Proof

We prove (a) ⇔\Leftrightarrow (b) ⇔\Leftrightarrow (c) for general random variables, using the technique of extending an identity from a generating class of sets to all Borel sets via Dynkin's π\pi-λ\lambda theorem, and then via a standard monotone-class / simple-function approximation argument for the expectation statement. We then prove (d) as a corollary using differentiation and integration.

(b) ⇒\Rightarrow (a). Take A=(−∞,x]A = (-\infty, x] and B=(−∞,y]B = (-\infty, y]. Then P(X∈A,Y∈B)=FX,Y(x,y)P(X \in A, Y \in B) = F_{X,Y}(x,y), P(X∈A)=FX(x)P(X\in A) = F_X(x), P(Y∈B)=FY(y)P(Y \in B) = F_Y(y), so (a) is the special case of (b) with these sets. This direction is immediate.

(a) ⇒\Rightarrow (b). This is the substantial direction; we use the π\pi-λ\lambda theorem (Dynkin's theorem). Step 1 (fix BB, vary AA over a π\pi-system). Fix y∈Ry \in \mathbb{R} and let B=(−∞,y]B = (-\infty, y]. Define the collection

Dy={A∈B(R):P(X∈A,Y≤y)=P(X∈A)P(Y≤y)}.\mathcal{D}_y = \{ A \in \mathcal{B}(\mathbb{R}) : P(X \in A, Y \le y) = P(X \in A) P(Y \le y) \}.

Let P={(−∞,x]:x∈R}\mathcal{P} = \{(-\infty, x] : x \in \mathbb{R}\}, a π\pi-system (closed under finite intersection, since (−∞,x1]∩(−∞,x2]=(−∞,min⁡(x1,x2)](-\infty,x_1]\cap(-\infty,x_2] = (-\infty, \min(x_1,x_2)]) that generates B(R)\mathcal{B}(\mathbb{R}). By hypothesis (a), P⊆Dy\mathcal{P} \subseteq \mathcal{D}_y. Step 2 (Dy\mathcal{D}_y is a λ\lambda-system). We check the three defining properties.

  • R∈Dy\mathbb{R} \in \mathcal{D}_y since P(X∈R,Y≤y)=P(Y≤y)=1⋅P(Y≤y)=P(X∈R)P(Y≤y)P(X \in \mathbb{R}, Y \le y) = P(Y\le y) = 1 \cdot P(Y \le y) = P(X \in \mathbb{R})P(Y\le y).
  • If A∈DyA \in \mathcal{D}_y, then for the complement AcA^c,
P(X∈Ac,Y≤y)=P(Y≤y)−P(X∈A,Y≤y)=P(Y≤y)(1−P(X∈A))=P(X∈Ac)P(Y≤y),P(X \in A^c, Y \le y) = P(Y \le y) - P(X \in A, Y \le y) = P(Y\le y)\big(1 - P(X\in A)\big) = P(X \in A^c)P(Y \le y),

using A∈DyA \in \mathcal{D}_y in the middle step. So Ac∈DyA^c \in \mathcal{D}_y.

  • If A1,A2,⋯∈DyA_1, A_2, \dots \in \mathcal{D}_y are pairwise disjoint with union AA, then {X∈An,Y≤y}\{X \in A_n, Y\le y\} are pairwise disjoint with union {X∈A,Y≤y}\{X \in A, Y \le y\}, so by countable additivity
P(X∈A,Y≤y)=∑nP(X∈An,Y≤y)=∑nP(X∈An)P(Y≤y)=P(X∈A)P(Y≤y),P(X \in A, Y\le y) = \sum_n P(X \in A_n, Y \le y) = \sum_n P(X \in A_n) P(Y \le y) = P(X \in A) P(Y \le y),

where the last equality uses countable additivity of P(X∈⋅)P(X \in \cdot) again. So A∈DyA \in \mathcal{D}_y. Hence Dy\mathcal{D}_y is a λ\lambda-system containing the π\pi-system P\mathcal{P}. By Dynkin's π\pi-λ\lambda theorem, Dy⊇σ(P)=B(R)\mathcal{D}_y \supseteq \sigma(\mathcal{P}) = \mathcal{B}(\mathbb{R}). Since trivially Dy⊆B(R)\mathcal{D}_y \subseteq \mathcal{B}(\mathbb{R}), we get Dy=B(R)\mathcal{D}_y = \mathcal{B}(\mathbb{R}). So for every Borel AA and every yy,

P(X∈A,Y≤y)=P(X∈A)P(Y≤y).(∗)P(X \in A, Y \le y) = P(X \in A) P(Y \le y). \tag{$\ast$}

Step 3 (fix AA, vary BB over the same argument). Now fix a Borel set AA satisfying (∗)(\ast) for all yy, and define

EA={B∈B(R):P(X∈A,Y∈B)=P(X∈A)P(Y∈B)}.\mathcal{E}_A = \{B \in \mathcal{B}(\mathbb{R}) : P(X\in A, Y \in B) = P(X \in A) P(Y \in B)\}.

By (∗)(\ast), EA\mathcal{E}_A contains the π\pi-system {(−∞,y]:y∈R}\{(-\infty,y] : y \in \mathbb{R}\}. The identical argument as in Step 2 (complements via subtraction, countable unions via disjoint additivity) shows EA\mathcal{E}_A is a λ\lambda-system, hence EA=B(R)\mathcal{E}_A = \mathcal{B}(\mathbb{R}) by Dynkin's theorem again. Since AA was an arbitrary Borel set satisfying (∗)(\ast) — and every Borel set does, by Step 2 — this proves (b) in full generality. ■\blacksquare (a)⇔\Leftrightarrow(b)

(b) ⇒\Rightarrow (c). First suppose g=1Ag = \mathbf{1}_A, h=1Bh = \mathbf{1}_B are indicators of Borel sets. Then

E[g(X)h(Y)]=E[1A(X)1B(Y)]=P(X∈A,Y∈B)=P(X∈A)P(Y∈B)=E[g(X)]E[h(Y)],E[g(X)h(Y)] = E[\mathbf{1}_A(X)\mathbf{1}_B(Y)] = P(X\in A, Y \in B) = P(X\in A)P(Y\in B) = E[g(X)]E[h(Y)],

using (b) in the third equality. By linearity of expectation, the identity extends to simple functions g=∑ici1Aig = \sum_i c_i \mathbf{1}_{A_i}, h=∑jdj1Bjh = \sum_j d_j \mathbf{1}_{B_j} (finite linear combinations of indicators of Borel sets):

E[g(X)h(Y)]=∑i,jcidj E[1Ai(X)1Bj(Y)]=∑i,jcidj E[1Ai(X)]E[1Bj(Y)]=E[g(X)]E[h(Y)].E[g(X)h(Y)] = \sum_{i,j} c_i d_j\, E[\mathbf{1}_{A_i}(X)\mathbf{1}_{B_j}(Y)] = \sum_{i,j} c_i d_j\, E[\mathbf{1}_{A_i}(X)]E[\mathbf{1}_{B_j}(Y)] = E[g(X)]E[h(Y)].

For general bounded measurable g,h≥0g,h \ge 0, choose increasing sequences of nonnegative simple functions gn↑gg_n \uparrow g and hn↑hh_n \uparrow h pointwise (standard simple-function approximation of measurable functions). Then gn(X)hn(Y)↑g(X)h(Y)g_n(X)h_n(Y) \uparrow g(X)h(Y) pointwise, and by the Monotone Convergence Theorem,

E[g(X)h(Y)]=lim⁡nE[gn(X)hn(Y)]=lim⁡nE[gn(X)]E[hn(Y)]=E[g(X)] E[h(Y)],E[g(X)h(Y)] = \lim_n E[g_n(X)h_n(Y)] = \lim_n E[g_n(X)]E[h_n(Y)] = E[g(X)]\,E[h(Y)],

where the last equality uses monotone convergence separately on each factor. For general bounded measurable g,hg,h, decompose g=g+−g−g = g^+ - g^-, h=h+−h−h = h^+-h^- into nonnegative parts and expand g(X)h(Y)=g+(X)h+(Y)−g+(X)h−(Y)−g−(X)h+(Y)+g−(X)h−(Y)g(X)h(Y) = g^+(X)h^+(Y) - g^+(X)h^-(Y) - g^-(X)h^+(Y) + g^-(X)h^-(Y); apply the nonnegative case to each of the four terms and use linearity of expectation (all terms are integrable since g,hg,h are bounded). This proves (c) for bounded g,hg,h; the extension to nonnegative unbounded g,hg,h follows the same monotone-convergence argument, and to integrable g,hg,h by the same decomposition provided integrability of the products holds (this is exactly the content of the parenthetical in (c)).

(c) ⇒\Rightarrow (b). Take g=1Ag = \mathbf{1}_A, h=1Bh = \mathbf{1}_B (bounded, measurable); (c) directly gives P(X∈A,Y∈B)=P(X∈A)P(Y∈B)P(X \in A, Y\in B) = P(X\in A)P(Y \in B), which is (b). ■\blacksquare (b)⇔\Leftrightarrow(c)

Proof of (d), discrete case. Suppose X,YX,Y discrete with countable ranges {xi},{yj}\{x_i\}, \{y_j\}. If (a)/(b) holds, apply (b) with A={xi},B={yj}A = \{x_i\}, B=\{y_j\} (singletons are Borel):

pX,Y(xi,yj)=P(X=xi,Y=yj)=P(X=xi)P(Y=yj)=pX(xi)pY(yj).p_{X,Y}(x_i,y_j) = P(X=x_i, Y=y_j) = P(X=x_i)P(Y=y_j) = p_X(x_i)p_Y(y_j).

Conversely, if pX,Y(xi,yj)=pX(xi)pY(yj)p_{X,Y}(x_i,y_j) = p_X(x_i)p_Y(y_j) for all i,ji,j, then for any Borel A,BA,B,

P(X∈A,Y∈B)=∑xi∈A∑yj∈BpX,Y(xi,yj)=(∑xi∈ApX(xi))(∑yj∈BpY(yj))=P(X∈A)P(Y∈B),P(X\in A, Y \in B) = \sum_{x_i \in A}\sum_{y_j \in B} p_{X,Y}(x_i,y_j) = \Big(\sum_{x_i\in A} p_X(x_i)\Big)\Big(\sum_{y_j \in B} p_Y(y_j)\Big) = P(X\in A)P(Y\in B),

which is (b), equivalent to (a).

Proof of (d), continuous case. Suppose (X,Y)(X,Y) has joint density fX,Yf_{X,Y}, i.e. FX,Y(x,y)=∫−∞x∫−∞yfX,Y(s,t) dt dsF_{X,Y}(x,y) = \int_{-\infty}^x\int_{-\infty}^y f_{X,Y}(s,t)\,dt\,ds. Suppose X,YX,Y have marginal densities fX,fYf_X, f_Y and fX,Y(s,t)=fX(s)fY(t)f_{X,Y}(s,t) = f_X(s)f_Y(t) a.e. Then

FX,Y(x,y)=∫−∞x∫−∞yfX(s)fY(t) dt ds=(∫−∞xfX(s) ds)(∫−∞yfY(t) dt)=FX(x)FY(y),F_{X,Y}(x,y) = \int_{-\infty}^x\int_{-\infty}^y f_X(s)f_Y(t)\,dt\,ds = \Big(\int_{-\infty}^x f_X(s)\,ds\Big)\Big(\int_{-\infty}^y f_Y(t)\,dt\Big) = F_X(x)F_Y(y),

by Fubini's theorem (the integrand is nonnegative, so Tonelli applies without further integrability hypotheses), giving (a). Conversely, suppose (a) holds and (X,Y)(X,Y) has a joint density fX,Yf_{X,Y}. Then XX has marginal density fX(s)=∫RfX,Y(s,t) dtf_X(s) = \int_{\mathbb R} f_{X,Y}(s,t)\,dt and likewise fYf_Y. From (a), for all x,yx,y,

∫−∞x∫−∞yfX,Y(s,t) dt ds=(∫−∞xfX(s) ds)(∫−∞yfY(t) dt)=∫−∞x∫−∞yfX(s)fY(t) dt ds,\int_{-\infty}^x \int_{-\infty}^y f_{X,Y}(s,t)\,dt\,ds = \Big(\int_{-\infty}^x f_X(s)\,ds\Big)\Big(\int_{-\infty}^y f_Y(t)\,dt\Big) = \int_{-\infty}^x\int_{-\infty}^y f_X(s)f_Y(t)\,dt\,ds,

where the right side used Fubini/Tonelli to write the product of two integrals as one double integral. Since this holds for every x,y∈Rx,y \in \mathbb{R}, differentiating both sides with respect to xx then yy (valid at every Lebesgue point of the respective integrands, i.e. Lebesgue-almost everywhere, by the Lebesgue differentiation theorem) gives

fX,Y(x,y)=fX(x)fY(y)for a.e. (x,y).f_{X,Y}(x,y) = f_X(x) f_Y(y) \quad \text{for a.e. } (x,y).

This completes the proof of (d), and hence of the theorem. ■\blacksquare

∎
CorollaryIndependence and functions

If XX and YY are independent, and ϕ,ψ:R→R\phi, \psi: \mathbb{R}\to\mathbb{R} are Borel measurable, then ϕ(X)\phi(X) and ψ(Y)\psi(Y) are independent.

Proof

For Borel sets C,D⊆RC, D \subseteq \mathbb{R},

P(ϕ(X)∈C, ψ(Y)∈D)=P(X∈ϕ−1(C), Y∈ψ−1(D))=P(X∈ϕ−1(C))P(Y∈ψ−1(D)),P(\phi(X)\in C,\ \psi(Y) \in D) = P(X \in \phi^{-1}(C),\, Y \in \psi^{-1}(D)) = P(X\in\phi^{-1}(C))P(Y\in\psi^{-1}(D)),

using characterization (b) of the theorem with A=ϕ−1(C)A=\phi^{-1}(C), B=ψ−1(D)B=\psi^{-1}(D) (both Borel since ϕ,ψ\phi,\psi are measurable). The right side equals P(ϕ(X)∈C)P(ψ(Y)∈D)P(\phi(X)\in C)P(\psi(Y)\in D), which is (b) for ϕ(X),ψ(Y)\phi(X),\psi(Y). ■\blacksquare

∎

Worked examples

Example 1: A discrete asymmetric joint distribution. Let X∈{1,2,3}X \in \{1,2,3\} and Y∈{0,5}Y \in \{0, 5\} have the joint probability mass function

Y=0Y=0Y=5Y=5row total
X=1X=10.100.100.150.150.250.25
X=2X=20.060.060.090.090.150.15
X=3X=30.240.240.360.360.600.60
col total0.400.400.600.601.001.00

Check independence via (d): we need pX,Y(x,y)=pX(x)pY(y)p_{X,Y}(x,y) = p_X(x) p_Y(y) for all nine (three times... here six) cells. pX(1)=0.25, pX(2)=0.15, pX(3)=0.60p_X(1) = 0.25,\ p_X(2)=0.15,\ p_X(3)=0.60; pY(0)=0.40, pY(5)=0.60p_Y(0)=0.40,\ p_Y(5)=0.60.

  • pX(1)pY(0)=0.25×0.40=0.10p_X(1)p_Y(0) = 0.25\times0.40 = 0.10 ✓ (matches table)
  • pX(1)pY(5)=0.25×0.60=0.15p_X(1)p_Y(5) = 0.25\times0.60 = 0.15 ✓
  • pX(2)pY(0)=0.15×0.40=0.06p_X(2)p_Y(0) = 0.15\times0.40 = 0.06 ✓
  • pX(2)pY(5)=0.15×0.60=0.09p_X(2)p_Y(5) = 0.15\times0.60 = 0.09 ✓
  • pX(3)pY(0)=0.60×0.40=0.24p_X(3)p_Y(0) = 0.60\times0.40 = 0.24 ✓
  • pX(3)pY(5)=0.60×0.60=0.36p_X(3)p_Y(5) = 0.60\times0.60 = 0.36 ✓

All six cells match, so XX and YY are independent despite the deliberately asymmetric (non-uniform, non-symmetric) marginals — independence is about factorization, not about the marginals looking "nice." As a further check via expectation factorization (c), take g(x)=x2g(x) = x^2, h(y)=eyh(y) = e^{y}:

E[X2Y] via factorization should equal E[X2] E[Y].E[X^2 Y] \text{ via factorization should equal } E[X^2]\,E[Y].

Compute E[X2]=12(0.25)+22(0.15)+32(0.60)=0.25+0.60+5.40=6.25E[X^2] = 1^2(0.25) + 2^2(0.15) + 3^2(0.60) = 0.25 + 0.60 + 5.40 = 6.25, and E[Y]=0(0.40)+5(0.60)=3.0E[Y] = 0(0.40)+5(0.60) = 3.0, so E[X2]E[Y]=18.75E[X^2]E[Y] = 18.75. Direct computation:

E[X2Y]=∑x,yx2y pX,Y(x,y)=1(5)(0.15)+4(5)(0.09)+9(5)(0.36)=0.75+1.8+16.2=18.75.E[X^2 Y] = \sum_{x,y} x^2 y\, p_{X,Y}(x,y) = 1(5)(0.15)+4(5)(0.09)+9(5)(0.36) = 0.75+1.8+16.2 = 18.75.

Matches, consistent with independence (as it must, by the theorem).

Example 2: A continuous example with genuinely non-symmetric densities on non-rectangular-looking but factorizable support. Let XX have density fX(x)=2xf_X(x) = 2x on (0,1)(0,1) (zero elsewhere), and YY have density fY(y)=3y2f_Y(y) = 3y^2 on (0,1)(0,1) (zero elsewhere) — both are valid densities since ∫012x dx=1\int_0^1 2x\,dx=1, ∫013y2 dy=1\int_0^1 3y^2\,dy=1. Define the joint density

fX,Y(x,y)=6xy2,0<x<1, 0<y<1.f_{X,Y}(x,y) = 6xy^2, \qquad 0<x<1,\ 0<y<1.

This is a product fX(x)fY(y)=(2x)(3y2)=6xy2f_X(x)f_Y(y) = (2x)(3y^2) = 6xy^2, so by part (d) of the theorem XX and YY are independent, and moreover we've constructed an independent pair by simply multiplying two arbitrary one-dimensional densities together — this is the standard method of building independent variables. Verify via a genuinely nontrivial event-factorization check (characterization (b)): let A=[12,1)A = [\tfrac12, 1), B=[0,13)B = [0,\tfrac13).

P(X∈A)=∫1/212x dx=[x2]1/21=1−14=34.P(X\in A) = \int_{1/2}^1 2x\,dx = \left[x^2\right]_{1/2}^1 = 1 - \tfrac14 = \tfrac34.
P(Y∈B)=∫01/33y2 dy=[y3]01/3=127.P(Y \in B) = \int_0^{1/3} 3y^2\,dy = \left[y^3\right]_0^{1/3} = \tfrac{1}{27}.
P(X∈A)P(Y∈B)=34⋅127=3108=136.P(X\in A)P(Y\in B) = \tfrac34 \cdot \tfrac{1}{27} = \tfrac{3}{108} = \tfrac{1}{36}.

Direct joint computation:

P(X∈A,Y∈B)=∫1/21∫01/36xy2 dy dx=(∫1/216x dx)(∫01/3y2 dy)=[3x2]1/21⋅[y33]01/3.P(X\in A, Y\in B) = \int_{1/2}^1\int_0^{1/3} 6xy^2\,dy\,dx = \left(\int_{1/2}^1 6x\,dx\right)\left(\int_0^{1/3} y^2\,dy\right) = \left[3x^2\right]_{1/2}^1 \cdot \left[\tfrac{y^3}{3}\right]_0^{1/3}.

Compute [3x2]1/21=3−34=94\left[3x^2\right]_{1/2}^1 = 3 - \tfrac34 = \tfrac94, and [y33]01/3=13⋅127=181\left[\tfrac{y^3}{3}\right]_0^{1/3} = \tfrac{1}{3}\cdot\tfrac{1}{27} = \tfrac{1}{81}. Product: 94⋅181=9324=136\tfrac94 \cdot \tfrac{1}{81} = \tfrac{9}{324} = \tfrac{1}{36}. Matches exactly, confirming independence numerically on an asymmetric pair of sets.

CounterexampleZero covariance does not imply independence

The theorem's characterization (c) shows independence forces E[XY]=E[X]E[Y]E[XY] = E[X]E[Y] for the specific choice g(x)=x,h(y)=yg(x)=x, h(y)=y — i.e., independence implies zero covariance. One might be tempted to think the converse holds: that uncorrelated implies independent. This is false, and dropping the full-generality requirement in (c) — using it only for the single pair (g,h)=(id,id)(g,h)=(\mathrm{id},\mathrm{id}) instead of all bounded measurable g,hg,h — is exactly the gap that breaks the implication. Let XX be uniform on {−1,0,1}\{-1, 0, 1\}, each with probability 13\tfrac13, and let Y=X2Y = X^2. Clearly YY is a deterministic (and highly dependent) function of XX: knowing XX tells you YY exactly. So X,YX, Y are about as dependent as two variables can be, and in particular not independent — e.g. P(X=0,Y=1)=0≠P(X=0)P(Y=1)=13⋅23P(X=0, Y=1) = 0 \ne P(X=0)P(Y=1) = \tfrac13\cdot\tfrac23. Yet compute the covariance. First,

E[X]=13(−1)+13(0)+13(1)=0.E[X] = \tfrac13(-1) + \tfrac13(0) + \tfrac13(1) = 0.

Next,

E[XY]=E[X3]=13(−1)3+13(0)3+13(1)3=13(−1+0+1)=0.E[XY] = E[X^3] = \tfrac13(-1)^3 + \tfrac13(0)^3 + \tfrac13(1)^3 = \tfrac13(-1+0+1) = 0.

So Cov(X,Y)=E[XY]−E[X]E[Y]=0−0⋅E[Y]=0\mathrm{Cov}(X,Y) = E[XY] - E[X]E[Y] = 0 - 0\cdot E[Y] = 0. Thus XX and YY are uncorrelated but not independent. The failure is precisely because characterization (c) requires the factorization E[g(X)h(Y)]=E[g(X)]E[h(Y)]E[g(X)h(Y)]=E[g(X)]E[h(Y)] for all bounded measurable g,hg,h, not merely the identity functions. Here, taking g(x)=x2g(x) = x^2 and h(y)=yh(y)=y exposes the dependence:

E[X2Y]=E[Y2]=13(0)4+13(1)4+13(1)4=23,E[X^2 Y] = E[Y^2] = \tfrac13(0)^4+\tfrac13(1)^4+\tfrac13(1)^4 = \tfrac23,

while E[X2]E[Y]=(23)(23)=49≠23E[X^2]E[Y] = \left(\tfrac23\right)\left(\tfrac23\right) = \tfrac49 \ne \tfrac23. This single failed instance of the "for all g,hg,h" clause certifies dependence, illustrating why the theorem demands the quantifier "for all bounded measurable g,hg,h" rather than just linear ones — weakening it to only linear g,hg,h gives a strictly weaker notion (uncorrelatedness), not independence.

Scope, edge cases & common traps

SituationWhat holdsWhat fails / must be checked
X,YX,Y independentE[XY]=E[X]E[Y]E[XY]=E[X]E[Y], Var(X+Y)=VarX+VarY\mathrm{Var}(X+Y)=\mathrm{Var}X+\mathrm{Var}Y, FX,Y=FXFYF_{X,Y}=F_XF_Y—
Cov(X,Y)=0\mathrm{Cov}(X,Y)=0Linear dependence removedDoes not imply independence (see counterexample)
Pairwise independent X1,…,XnX_1,\dots,X_n (n≥3n\ge3)Each pair factorsMutual independence can fail (classic example below)
X,YX,Y jointly Gaussian, uncorrelatedIndependence does hold (special extra structure of the multivariate normal)Only true because of the specific joint-Gaussian form; not a general fact
Independent X,YX,Y, then ϕ(X),ψ(Y)\phi(X),\psi(Y)Independent for any Borel ϕ,ψ\phi,\psiRequires ϕ,ψ\phi,\psi measurable; always true otherwise

A standard example showing pairwise independence does not imply mutual independence: let X1,X2X_1, X_2 be independent fair coin flips valued in {0,1}\{0,1\}, and set X3=X1⊕X2X_3 = X_1 \oplus X_2 (XOR, i.e. X3=1X_3 = 1 iff X1≠X2X_1 \ne X_2). One can check every pair among X1,X2,X3X_1,X_2,X_3 is independent (each pair reproduces the uniform product distribution on {0,1}2\{0,1\}^2), yet the triple is not mutually independent because X3X_3 is a deterministic function of (X1,X2)(X_1,X_2): P(X1=0,X2=0,X3=0)=14≠P(X1=0)P(X2=0)P(X3=0)=18P(X_1=0,X_2=0,X_3=0) = \tfrac14 \ne P(X_1=0)P(X_2=0)P(X_3=0) = \tfrac18.

The mathematician behind it

Origins · 1 minAndrei Nikolaevich Kolmogorov and how this was discovered
AN
Andrei Nikolaevich Kolmogorov
1903–1987 · Soviet Union, 20th-century foundations of probability

Kolmogorov placed probability theory on rigorous measure-theoretic foundations in his 1933 monograph Grundbegriffe der Wahrscheinlichkeitsrechnung, defining a probability space as a measure space (Ω,F,P)(\Omega,\mathcal F,P) with total mass one. Within this framework he gave the modern definition of independence as factorization of joint probabilities over σ\sigma-algebras, replacing older, informally stated notions. His axiomatic approach unified previously disparate results — laws of large numbers, zero-one laws, strong limit theorems — under a single rigorous language, and independence became the structural hypothesis on which nearly all of that machinery rests.

History
- 1900 — Hilbert's sixth problem calls for an axiomatic treatment of probability, motivating later foundational work.
- 1909 — Borel proves the earliest strong law of large numbers for coin tosses, implicitly using independence of tosses.
- 1933 — Kolmogorov publishes the Grundbegriffe, giving the modern measure-theoretic definition of independence via σ\sigma-algebras.
- 1933 — Kolmogorov's zero-one law is established, sharpening consequences of independence for tail events.
- 1950s — Independence becomes the standard hypothesis underlying the classical central limit theorem and law of large numbers proofs taught today.
Is independence the same as "uncorrelated"?

No. Independence is a statement about the entire joint distribution: every event about XX must be probabilistically unrelated to every event about YY. Uncorrelatedness only says the linear association measured by E[XY]−E[X]E[Y]E[XY]-E[X]E[Y] is zero. Independence implies uncorrelatedness (take g,hg,h to be the identity in the expectation-factorization criterion), but the converse fails whenever there is nonlinear dependence, as in the counterexample Y=X2Y=X^2 with XX symmetric around zero. The only broad exception is jointly Gaussian random variables, where uncorrelated and independent do coincide, because of the special algebraic form of the multivariate normal density.

Why do we need the joint CDF rather than just the two marginal CDFs to define independence?

Two marginal distributions never determine the joint distribution by themselves; infinitely many joint distributions share the same pair of marginals (this is the content of copula theory). Independence is precisely the strongest possible statement pinning the joint distribution down uniquely as the product measure. Stating independence via the joint CDF (or joint density/mass function) directly encodes this pinning-down; marginals alone carry no information about dependence structure.

Does independence of XX and YY imply independence of X2X^2 and Y3Y^3?

Yes. This is exactly the corollary proven above: if X⊥YX \perp Y and ϕ,ψ\phi,\psi are Borel measurable functions, then ϕ(X)\phi(X) and ψ(Y)\psi(Y) are independent. Applying this with ϕ(x)=x2\phi(x)=x^2 and ψ(y)=y3\psi(y)=y^3 gives X2⊥Y3X^2 \perp Y^3. More generally, independence is preserved under applying separate measurable transformations to each variable, since preimages of Borel sets under measurable functions are still Borel, which is all the proof needs.

Can three variables be independent in pairs but not all together?

Yes, and this is a common trap. The canonical example uses two independent fair coins X1,X2∈{0,1}X_1, X_2 \in \{0,1\} and their XOR X3=X1⊕X2X_3 = X_1 \oplus X_2. Every pair among {X1,X2,X3}\{X_1,X_2,X_3\} is independent, since each pair reproduces the uniform distribution on {0,1}2\{0,1\}^2. But the triple is not mutually independent: X3X_3 is completely determined by X1,X2X_1,X_2, so P(X1=0,X2=0,X3=0)=1/4P(X_1=0,X_2=0,X_3=0)=1/4 while the product of the three individual probabilities is 1/81/8.

How is independence used to compute the distribution of a sum X+YX+Y?

When X,YX,Y are independent with densities fX,fYf_X, f_Y, the density of X+YX+Y is the convolution fX+Y(z)=∫fX(x)fY(z−x) dxf_{X+Y}(z) = \int f_X(x) f_Y(z-x)\,dx. This formula relies critically on independence: it comes from writing P(X+Y≤z)=∫P(Y≤z−x)fX(x) dxP(X+Y\le z) = \int P(Y \le z-x) f_X(x)\,dx, using P(Y≤z−x∣X=x)=P(Y≤z−x)P(Y \le z-x \mid X=x) = P(Y\le z-x), which is exactly independence. Without independence, no such simple formula exists in general, and this convolution identity is developed fully in Sums of Random Variables and Convolution.

Why does the definition use the CDF instead of the density or mass function directly?

Not every random variable has a probability density (continuous random variables) or is discrete; some distributions are neither (e.g. mixtures, or singular continuous distributions like the Cantor distribution). The CDF is always well-defined for any random variable, so defining independence via CDF factorization gives a definition with no restriction on the type of distribution. The density/mass factorization criteria are then derived as convenient equivalent tests that apply specifically when those objects exist, as shown in part (d) of the theorem.

Before this

Where this leads