Independence

Independence captures the intuitive idea that knowing whether one event happened does not change the probability we assign to another. Formally, events $A$ and $B$ are independent if the probability of both occurring equals the product of their individual probabilities. This concept is foundational: it lets us multiply probabilities, build complex models from simple parts, and justifies enormous swaths of statistics (random samples, the Central Limit Theorem, confidence intervals). A surprising fact is that independence is not the same as "having nothing to do with each other" causally — two events can be statistically independent while being deeply physically linked, and conversely two unrelated-seeming events can be dependent.

The Core Insight

Independence means multiplication works. When events are independent, the probability of a combination is the product of the parts — no correction factor, no information leaking between them. Equivalently, conditioning on one event leaves the other's probability untouched: $P(A\mid B)=P(A)$. All of the subject's power flows from this single multiplicative rule.

Formal Statement

Independence of Two Events

Let $(\Omega ,\mathcal{F},P)$ be a probability space with events $A,B\in \mathcal{F}$. The events $A$ and $B$ are independent if and only if

Here:

• $\Omega$ = the sample space (set of all outcomes),
• $\mathcal{F}$ = the collection of events (subsets of $\Omega$),
• $P$ = the probability measure, $P:\mathcal{F}\to [0,1]$,
• $A\cap B$ = the event "both $A$ and $B$ occur,"
• $P(A)$, $P(B)$ = the marginal probabilities of each event.

Conditional Form

If $P(B)>0$, independence is equivalent to

where $P(A\mid B)$ is the conditional probability of $A$ given that $B$ has occurred. This says: learning $B$ happened does not revise the probability of $A$.

Mutual Independence of Several Events

Events $A_1,A_2,\dots ,A_n$ are mutually independent if for every subset $\{i_1,\dots ,i_k\}\subseteq \{1,\dots ,n\}$,

Independence of Random Variables

Random variables $X$ and $Y$ are independent if for all real numbers $x,y$,

For discrete variables this is equivalent to the joint probability mass function factoring, and for continuous variables to the joint density factoring:

Consequences of Independence

If $X$ and $Y$ are independent random variables, then:

Why It Holds / Derivation

Conditional Form Equivalence

We show equation [1] $\iff$ equation [2] when $P(B)>0$.

1. By the definition of conditional probability, $P(A\mid B)=\frac{P(A\cap B)}{P(B)}$.
2. Assume independence, equation [1]: $P(A\cap B)=P(A)P(B)$.
3. Substitute into step 1: $P(A\mid B)=\frac{P(A)P(B)}{P(B)}=P(A)$, giving equation [2].
4. Conversely, assume $P(A\mid B)=P(A)$. Multiply both sides of step 1's definition by $P(B)$: $P(A\cap B)=P(A\mid B)P(B)=P(A)P(B)$, recovering equation [1].
5. Therefore the two characterizations are equivalent. $■$

Multiplicativity of Expectation

We derive equation [6] for independent continuous variables.

1. By definition, $E[XY]={\iint }_{{\mathbb{R}}^2}xy{f}_{X,Y}(x,y)dxdy$.
2. By independence, equation [5] gives $f_{X,Y}(x,y)=f_X(x)f_Y(y)$.
3. Substitute: $E[XY]=\iint xy{f}_X(x)f_Y(y)dxdy$.
4. The integrand separates: $=\left(\int x{f}_X(x)dx\right)\left(\int y{f}_Y(y)dy\right)$.
5. Each factor is a marginal expectation: $=E[X]E[Y]$. $■$

Andrei Kolmogorov | 1903–1987

The Soviet mathematician Andrei Kolmogorov gave probability theory its rigorous axiomatic foundation in his 1933 monograph Grundbegriffe der Wahrscheinlichkeitsrechnung. Within that framework, independence is not an afterthought but a defining structural concept — Kolmogorov elevated it to a precise measure-theoretic notion via the factorization of probabilities. His treatment unified centuries of informal "multiplying chances" into the clean definition used today, and his zero-one law shows how independence governs the behaviour of infinite sequences of random events.

Building Intuition

Imagine two separate coin flips. Knowing the first landed heads tells you absolutely nothing about the second — that's independence. The "multiply the probabilities" rule works because each outcome of the first coin gets paired with each outcome of the second, with no preference.

Think of a grid: the rows are outcomes of $A$, the columns are outcomes of $B$. Independence means the probability of any cell is just (row probability) × (column probability) — the grid is "rectangular," with no clumping. Dependence warps the grid: some cells get extra weight, some get less, and learning the column shifts your belief about the row.

A useful mental check: if I tell you $B$ happened, does your bet on $A$ change? If not, they're independent.

Why physicists care

In statistical mechanics, independence of particle states underlies the factorization of the partition function and the multiplicative structure of probabilities for non-interacting systems. When particles don't interact, their joint configuration probability factorizes exactly as in equation [3] — and the breakdown of independence (correlations) is precisely what signals interactions, phase transitions, and entanglement in quantum systems.

Worked Examples

Two Dice

1. $P(A)=\frac{1}{6}$ (one face out of six).
2. $P(B)=\frac{3}{6}=\frac{1}{2}$ (faces 2, 4, 6).
3. The dice are physically separate, so we test equation [1]. Count outcomes: $A\cap B$ requires first die = 4 (1 way) and second die even (3 ways) = 3 outcomes out of 36.
4. $P(A\cap B)=\frac{3}{36}=\frac{1}{12}$.
5. Check: $P(A)P(B)=\frac{1}{6}\cdot \frac{1}{2}=\frac{1}{12}$. ✓ Equal, so independent.

Drawing Cards Without Replacement

1. $P(A)=\frac{4}{52}=\frac{1}{13}$.
2. $P(B)$ overall (by symmetry / law of total probability) $=\frac{4}{52}=\frac{1}{13}$.
3. Compute $P(A\cap B)$: $P(A)\cdot P(B\mid A)=\frac{4}{52}\cdot \frac{3}{51}=\frac{12}{2652}=\frac{1}{221}$.
4. Test equation [1]: $P(A)P(B)=\frac{1}{13}\cdot \frac{1}{13}=\frac{1}{169}$.
5. Since $\frac{1}{221}\ne \frac{1}{169}$, the events are dependent. Drawing one Ace lowers the chance of a second.

Probability grid: independent vs. dependent events

Common Misconceptions

Independent ≠ Mutually Exclusive

A frequent error is confusing independence with mutual exclusivity (disjointness). Mutually exclusive events $A$ and $B$ satisfy $P(A\cap B)=0$ — they cannot both happen. But if both have positive probability, this forces dependence: knowing $A$ occurred guarantees $B$ did not, so $P(B\mid A)=0\ne P(B)$. Mutually exclusive events with nonzero probability are about as dependent as events can be.

Pairwise ≠ Mutual Independence

Equation [3] demands the product rule for all subsets. Three events can be pairwise independent (every pair satisfies equation [1]) yet fail to be mutually independent, because the triple product rule breaks. Always check the full hierarchy of subsets when more than two events are involved.

Pairwise but not Mutually Independent

Toss two fair coins. Define:

• $A$ = "first coin is heads," $P(A)=\frac{1}{2}$
• $B$ = "second coin is heads," $P(B)=\frac{1}{2}$
• $C$ = "the two coins match (both H or both T)," $P(C)=\frac{1}{2}$

Each pair is independent: e.g. $P(A\cap C)=P(\text{HH})=\frac{1}{4}=P(A)P(C)$, and similarly for the others. But $A\cap B\cap C=\{\text{HH}\}$, so $P(A\cap B\cap C)=\frac{1}{4}$, whereas $P(A)P(B)P(C)=\frac{1}{8}$. Since $\frac{1}{4}\ne \frac{1}{8}$, the three are not mutually independent — indeed, knowing $A$ and $B$ completely determines $C$.

Uncorrelated ≠ Independent

Independence implies zero correlation (equation [6] forces $\mathrm{Cov}=0$), but the converse fails. Two variables can have zero covariance yet be strongly dependent — for example, $X$ uniform on $[-1,1]$ and $Y=X^2$ have $\mathrm{Cov}(X,Y)=0$ but $Y$ is completely determined by $X$. Correlation only measures linear association.

Connections

Independence is the keystone linking many areas. It is defined through Conditional Probability (equation [2]) and gives the clean form of Bayes' Theorem when priors and likelihoods factor. It powers the Binomial Distribution, which counts successes in independent trials, and is the crucial hypothesis behind the Central Limit Theorem and the Law of Large Numbers. In statistics it underlies the assumption of Random Sampling. The failure of independence is quantified by Covariance and Correlation, and its multivariate structure appears in the Joint Probability Distribution.

Real-World Applications

• Genetics (Mendelian inheritance): The independent assortment of alleles lets geneticists multiply probabilities to predict offspring genotype ratios (e.g. the 9:3:3:1 dihybrid ratio).
• Reliability engineering: A system of components in parallel fails only if all fail; assuming independent failures, the system failure probability is the product of individual failure probabilities.
• Cryptography: Secure random key generation and the "one-time pad" rely on bits being independent of the message, guaranteeing perfect secrecy.
• Insurance and actuarial science: Pooling many independent policyholder risks makes aggregate losses predictable via the Law of Large Numbers, setting premiums.
• Naïve Bayes classifiers: This machine-learning algorithm assumes features are conditionally independent given the class, drastically simplifying text and spam classification.
• Quality control: Independent random sampling from a production line lets engineers estimate defect rates and build control charts.
• Quantum mechanics: The failure of independence between particles (entanglement) is a measurable, exploitable resource for quantum computing and cryptography.

Historical Development

• 1654 — Pascal and Fermat correspond on games of chance, implicitly multiplying probabilities of independent rolls.
• 1657 — Christiaan Huygens publishes De ratiociniis in ludo aleae, the first printed probability treatise, using product reasoning.
• 1713 — Jacob Bernoulli's Ars Conjectandi (posthumous) develops the Law of Large Numbers for independent Bernoulli trials.
• 1718 — Abraham de Moivre's The Doctrine of Chances states the multiplication rule for independent events explicitly.
• 1733 — De Moivre derives the normal approximation to the binomial, an early Central Limit result for independent trials.
• 1812 — Pierre-Simon Laplace's Théorie analytique des probabilités systematizes conditional and independent probability.
• 1900 — Karl Pearson develops correlation, sharpening the distinction between dependence and independence.
• 1933 — Andrei Kolmogorov axiomatizes probability, giving independence its modern measure-theoretic definition (equation [1]) and proving the zero-one law.
• 1950s onward — Independence assumptions become central to mathematical statistics, information theory (Shannon), and stochastic processes.

Subtleties & Limits

Independence is mathematically crisp but subtle in application. Testing for independence from finite data is genuinely hard: standard tests (chi-square, distance correlation) can fail to detect complex nonlinear dependence, and "independence" is never proven, only failed-to-reject. In high-dimensional settings, distinguishing true independence from spurious correlations is an active research area. Deeper still, the notion of conditional independence — central to graphical models, causal inference, and Bayesian networks — generates rich combinatorial structure, and characterizing which conditional independence relations are simultaneously realizable remains partly open. Finally, in foundations, the relationship between statistical independence and causal independence is philosophically and practically unresolved: independence is symmetric and observational, while causation is directional, so correlation/dependence can never by itself establish or refute a causal link.

See Also