The Law of Total Probability

In statistics, the Law of Total Probability underlies computing marginal distributions from joint or conditional models, such as mixture models where a population is partitioned into latent subgroups (clusters, risk classes) with different conditional behavior. In medical diagnostics and epidemiology, it computes the overall prevalence of a condition or the overall sensitivity of a screening protocol across demographic strata, and is the direct setup for the denominator in Bayes' Theorem when inverting test results into diagnoses. In computer science and information theory, it underlies channel capacity computations, where the average error probability over transmitted symbols is a partition-weighted sum of per-symbol error probabilities. In actuarial science and insurance, total claim probabilities are computed by partitioning policyholders into risk classes, weighting each class's claim probability by its share of the portfolio. In cryptography and randomized algorithms, total probability arguments bound the overall success or failure probability of an algorithm by conditioning on which random branch or key was used. In physics and engineering reliability, it computes overall failure probabilities of systems partitioned by which component or failure mode is triggered, feeding directly into fault-tree analysis.

Ask the tutor: How does the Law of Total Probability turn into the denominator of Bayes' Theorem when we want to invert conditional probabilities?

Picture the sample space $\Omega$ as a rectangular region cut into disjoint vertical strips $A_1, A_2, A_3$, each strip representing one "scenario" — say, which of three machines produced a randomly chosen item. The event $B$ (say, "the item is defective") is some blob overlapping several strips. Within each strip, the fraction of that strip covered by $B$ is the conditional probability $P(B\mid A_i)$. The width of the strip is $P(A_i)$. The total area covered by $B$ — which is $P(B)$ — is obtained by adding up, strip by strip, the area of $B$ within that strip, and area within a strip is exactly (fraction covered) $\times$ (strip width). That is precisely $\sum_i P(B\mid A_i)P(A_i)$. This is why the law can be read as a weighted average: $P(B)$ is a convex combination of the conditional probabilities $P(B\mid A_i)$, with weights $P(A_i)$ that sum to 1. If one supplier dominates production, its defect rate dominates the overall rate; if all suppliers contribute equally, the overall rate is a simple average of the three defect rates.

Example 1 — Mixed supplier shipment (finite partition, asymmetric weights). A warehouse stocks resistors from three suppliers. Supplier $A_1$ provides 50% of stock with a 4% defect rate; supplier $A_2$ provides 30% of stock with a 7% defect rate; supplier $A_3$ provides the remaining 20% with a 1% defect rate. A resistor is chosen uniformly at random from the warehouse. Let $D$ be the event it is defective. The events $A_1, A_2, A_3$ (which supplier the resistor came from) form a partition: they are mutually exclusive (a resistor has exactly one supplier) and exhaustive ($P(A_1) + P(A_2) + P(A_3) = 0.5 + 0.3 + 0.2 = 1$). We are given

By the Law of Total Probability,

Compute each term: $0.04 \times 0.5 = 0.020$; $0.07 \times 0.3 = 0.021$; $0.01 \times 0.2 = 0.002$. Summing,

So 4.3% of the warehouse's resistors are defective overall — a figure strictly between the extremes (1% and 7%) as it must be, since it is a weighted average, but skewed toward Supplier $A_1$'s 4% because that supplier contributes the largest share.

Example 2 — A countably infinite partition (geometric number of trials). A machine attempts to complete a task repeatedly; on each attempt it succeeds independently with probability $p = 0.3$. Let $N$ be the attempt number on which the first success occurs, so $N$ is geometric on $\{1, 2, 3, \ldots\}$ with

This gives a countably infinite partition $\{A_n\}_{n=1}^\infty$ of the sample space, $A_n = \{N = n\}$: exactly one value of $N$ occurs, so the $A_n$ are disjoint, and $\sum_{n=1}^{\infty} P(N=n) = 1$ (a standard geometric series check: $\sum_{n=1}^\infty (0.7)^{n-1}(0.3) = 0.3 \cdot \frac{1}{1-0.7} = 0.3 \cdot \frac{1}{0.3} = 1$), so they are exhaustive. Suppose that once the machine finally succeeds on attempt $N=n$, it then produces a usable output with probability $r_n = \frac{1}{n+1}$ (later successes are "rustier," modeling wear), i.e. $P(U \mid A_n) = \frac{1}{n+1}$, where $U$ is the event of a usable output. We want $P(U)$, the overall (marginal) probability the eventual output is usable. By the Law of Total Probability (countable partition case),

Write $q = 0.7$. Then

Reindex with $k = n+1$ (so $n = k-1$, $k$ runs from $2$ to $\infty$):

Recall the Taylor series $-\ln(1-q) = \sum_{k=1}^{\infty} \frac{q^k}{k}$ for $|q| < 1$, so $\sum_{k=2}^\infty \frac{q^k}{k} = -\ln(1-q) - q$. With $q = 0.7$:

Then

Finally,

So despite each individual conditional probability $P(U\mid A_n) = \frac{1}{n+1}$ shrinking toward 0, the overall probability of a usable output is about 30.9%, dominated by the early terms ($n=1$ contributes $P(U\mid A_1)P(A_1) = \frac12 \cdot 0.3 = 0.15$, already about half the total) because early successes are by far the most likely outcomes under the geometric distribution.