Independence of Events

Independence is the load-bearing assumption behind the Binomial Distribution: a sequence of $n$ trials, each succeeding with probability $p$, yields a binomial count only because the trials are modeled as mutually independent, so that the probability of any specific sequence of successes and failures factors into a product of $p$'s and $(1-p)$'s. The same idea underlies the Poisson approximation to rare independent events, the multiplication rule used throughout combinatorial probability, and the entire notion of a random sample in statistics, where "independent and identically distributed" (i.i.d.) observations are the default assumption behind estimators, confidence intervals, and hypothesis tests. In statistical inference, checking whether real data can plausibly be modeled as independent (versus, say, autocorrelated time series data) is a first-order modeling decision. In cryptography, independence of key bits and independence of plaintext from ciphertext (in appropriate senses) are central to formal definitions of security; Shannon's notion of "perfect secrecy" is phrased exactly as a probabilistic independence condition between message and ciphertext. In statistical mechanics and coding theory, independence of noise across channel uses or independence of particle states underlies both channel capacity calculations and the derivation of macroscopic laws from microscopic randomness. In computer science, randomized algorithms and hashing schemes are analyzed via carefully constructed families of "pairwise independent" or "$k$-wise independent" random variables — a direct generalization of exactly the pairwise/mutual distinction proved in this article, deliberately exploited because pairwise independence is cheaper to achieve while still sufficing for variance calculations via the identity $\mathrm{Var}(X+Y) = \mathrm{Var}(X)+\mathrm{Var}(Y)$ for independent $X, Y$. Finally, independence is the conceptual seed for the definition of independent Random Variables, which extends the event-level product condition to full distributional factorization and underlies nearly every model in applied probability.

Ask the tutor: Can you show me an example where the full triple product $P(A_1\cap A_2\cap A_3) = P(A_1)P(A_2)P(A_3)$ holds exactly, yet the events are still not mutually independent because a pair fails?

The defining equation $P(A \cap B) = P(A)P(B)$ is best understood through Conditional Probability. Recall that when $P(B) > 0$,

If $A$ and $B$ are independent, this becomes $P(A \mid B) = P(A)$: learning that $B$ occurred leaves your assessment of $A$ completely unchanged. This is the operational meaning of independence — it is a statement about information, not about physical mechanism. Two events can be "independent" in this technical sense purely by numerical coincidence, with no causal story behind it at all; conversely, events that "feel" unrelated can fail to be independent if the numbers don't cooperate. A geometric picture helps. Think of $\Omega$ as a unit square with area representing probability, and model $A$ as a vertical strip and $B$ as a horizontal strip. The intersection $A \cap B$ is then a rectangle, and its area is automatically the product of the strip widths — independence is "built in" to this picture. This is exactly why independence is often modeled using product spaces: rolling a die and flipping a coin are naturally independent because the sample space itself is a Cartesian product, and each event depends on only one coordinate.

Example 1 (Cards: a genuinely asymmetric pair). Draw one card uniformly at random from a standard 52-card deck. Let

There are 13 hearts, so $P(A) = 13/52 = 1/4$. There are 12 face cards ($3$ ranks $\times$ $4$ suits), so $P(B) = 12/52 = 3/13$. The event $A \cap B$ is "heart and face card": there are exactly 3 such cards (J♥, Q♥, K♥), so

Now check the product:

Since $P(A \cap B) = P(A)P(B)$, the events $A$ and $B$ are independent. This is a genuine (if slightly surprising) fact about the structure of a deck: suit and face-status are independent, even though there is no "randomizing mechanism" separating them — it is a consequence of the deck having the same number of face cards in every suit, which forces the factorization by symmetry of counting. This illustrates the point made in the Intuition section: independence here is a numerical fact, not a causal one. As a further check via part (a) of the theorem, compute the conditional probability directly:

confirming independence without reference to the product formula. Now consider a related but non-independent pair: let $C = \{\text{the card is a spade}\}$ and $D = \{\text{the card is red}\}$ (hearts or diamonds). Then $P(C) = 1/4$, $P(D) = 1/2$, but $C \cap D = \varnothing$ since no card is both a spade and red, so $P(C \cap D) = 0 \ne \frac14 \cdot \frac12 = \frac18$. Here $C$ and $D$ are strongly (negatively) dependent — knowing a card is a spade tells you for certain it is not red. This contrast sharpens the definition: independence is a precise numerical balance, and "unrelated-sounding" events like suit-color and specific-suit can fail it dramatically.

Example 2 (Three fair coin flips: verifying full mutual independence directly from a model). Flip a fair coin three times independently in the everyday sense, modeled by the sample space $\Omega = \{H,T\}^3$ of 8 equally likely outcomes, each with probability $1/8$. Define

This example is deliberately asymmetric: $A$ and $B$ concern individual coordinates, while $C$ is a nontrivial function of all three coordinates, and we must check all four required product identities (three pairs and one triple) from the definition of mutual independence — nothing is assumed. List the 8 outcomes with their membership:

Count: $A$ occurs in 4 outcomes, so $P(A) = 4/8 = 1/2$. Likewise $P(B) = 1/2$. $C$ occurs in HHH, HHT, HTH, THH — 4 outcomes — so $P(C) = 1/2$. Pair $A,B$: $A \cap B = \{HHH, HHT\}$, so $P(A \cap B) = 2/8 = 1/4 = (1/2)(1/2)$. ✓. Pair $A,C$: $A \cap C = \{HHH, HHT, HTH\}$, so $P(A \cap C) = 3/8$. But $P(A)P(C) = (1/2)(1/2) = 1/4 = 2/8$. Since $3/8 \ne 2/8$, $A$ and $C$ are not independent. This already shows $A, B, C$ are not mutually independent (mutual independence requires every pair to be independent), and it is a useful cautionary computation: $C$ is built from $A$ and $B$ (among others), so it is intuitively unsurprising that $C$ carries information about $A$ — knowing "at least two heads" makes "flip 1 is heads" more likely than $1/2$. Indeed $P(A \mid C) = P(A\cap C)/P(C) = (3/8)/(1/2) = 3/4 > 1/2 = P(A)$, confirming the failure quantitatively: seeing $C$ occur genuinely raises your assessment of $A$. Contrast this with the event $D = \{\text{flip 3 is } H\}$, which depends on a coordinate untouched by $A$ and $B$. One checks $A \cap D = \{HHH, HTH\}$ so $P(A\cap D) = 2/8 = 1/4 = P(A)P(D)$; similarly for $B,D$; and

so $A, B, D$ are mutually independent — as expected, since they depend on three physically and probabilistically separate coordinates of the product space. The moral: mutual independence must be verified equation by equation; it is not conferred automatically just because events look like they come from "different parts" of an experiment, as the $A, B, C$ case shows.