Total Probability Theorem
The Total Probability Theorem is one of the most powerful tools in probability theory. It allows us to compute the probability of a complex event by breaking it into simpler, non-overlapping cases. Imagine you want to find the probability of drawing a red card from a shuffled deck, but the deck was chosen randomly from several different decks — the Total Probability Theorem gives us a systematic way to combine the conditional probabilities from each case into a single overall answer. The key idea rests on the concept of a partition. A partition of the sample space S is a collection of mutually exclusive and exhaustive events B₁, B₂, …, Bₙ — meaning no two of them overlap, and together they cover every possible outcome. Once we have such a partition, any event A can be written as the union of pieces: A occurring together with B₁, A occurring together with B₂, and so on. Because these pieces don't overlap, we can add their probabilities, leading to a clean formula. This theorem appears throughout science and everyday reasoning. Medical tests, quality-control checks in factories, weather forecasting, and machine-learning classifiers all use the Total Probability Theorem under the hood. Mastering it builds the foundation for Bayes' Theorem, which inverts conditional probabilities and is central to modern data science and statistics.
- Define a Partition of the Sample Space
- Decompose Event A Using the Partition
- Apply the Multiplication Rule to Each Piece
- Interpret the Weighted Average
- Check That Probabilities Sum to 1
Define a Partition of the Sample Space
Choose events B₁, B₂, …, Bₙ that are mutually exclusive: Bᵢ ∩ Bⱼ = ∅ whenever i ≠ j. They must also be exhaustive: B₁ ∪ B₂ ∪ … ∪ Bₙ = S. This means every outcome in S falls into exactly one Bᵢ. A classic example: a factory has three machines (B₁, B₂, B₃) producing all of its widgets — every widget comes from exactly one machine, so these three events partition the sample space of all widgets.
Decompose Event A Using the Partition
For any event A, we can write A = (A ∩ B₁) ∪ (A ∩ B₂) ∪ … ∪ (A ∩ Bₙ). Each piece A ∩ Bᵢ represents 'A occurred AND the outcome was in Bᵢ.' Because the Bᵢ are mutually exclusive, so are the pieces A ∩ Bᵢ. By the Addition Rule for mutually exclusive events, P(A) = P(A ∩ B₁) + P(A ∩ B₂) + … + P(A ∩ Bₙ).
Apply the Multiplication Rule to Each Piece
Recall the Multiplication Rule: P(A ∩ Bᵢ) = P(A | Bᵢ) · P(Bᵢ). Here P(A | Bᵢ) is the conditional probability of A given that Bᵢ occurred. Substituting into the sum gives the Total Probability Theorem: P(A) = Σᵢ P(A | Bᵢ) · P(Bᵢ). Each term weights the probability of A given scenario Bᵢ by how likely that scenario is.
Interpret the Weighted Average
The formula is a weighted average of conditional probabilities. The weight of each conditional probability P(A | Bᵢ) is the prior probability P(Bᵢ) of that scenario. If one scenario is very unlikely (small P(Bᵢ)), it contributes little to the total, even if A is very probable given that scenario. Conversely, a likely scenario (large P(Bᵢ)) carries heavy influence on the overall probability of A.
Check That Probabilities Sum to 1
A useful sanity check: since B₁, …, Bₙ form a partition, P(B₁) + P(B₂) + … + P(Bₙ) = 1. If all the conditional probabilities P(A | Bᵢ) were equal to some constant c, the formula would give P(A) = c · 1 = c, which makes intuitive sense — if A is equally likely regardless of which Bᵢ occurred, the unconditional probability equals that common value.
The Key Equation
- P(A) — Total (unconditional) probability of A (dimensionless, in [0, 1]): The overall probability of event A across the entire sample space.
- B_i — Partition event i (event): The i-th element of a partition of the sample space; the Bᵢ are mutually exclusive and exhaustive.
- P(B_i) — Prior probability of Bᵢ (dimensionless, in [0, 1]): The probability that outcome falls in scenario Bᵢ; these must sum to 1 across all i.
- P(A \mid B_i) — Conditional probability of A given Bᵢ (dimensionless, in [0, 1]): The probability that A occurs, given that we know the outcome is in Bᵢ.
- n — Number of partition elements (positive integer): The total number of mutually exclusive and exhaustive events forming the partition.
Worked Example
Problem. A factory has three machines producing light bulbs. Machine B₁ produces 50% of all bulbs, Machine B₂ produces 30%, and Machine B₃ produces 20%. The defect rates are: P(defective | B₁) = 0.02, P(defective | B₂) = 0.05, P(defective | B₃) = 0.04. A bulb is chosen at random from the day's production. What is the probability it is defective?
Answer. P(defective) = 0.033, or 3.3%
Origins · 1 minThomas Bayes and how this was discovered
Thomas Bayes was a Presbyterian minister and mathematician whose posthumously published essay introduced the idea of inverse probability — computing the probability of a cause given an observed effect. The Total Probability Theorem is the essential stepping stone to Bayes' Theorem, P(Bᵢ|A) = P(A|Bᵢ)·P(Bᵢ)/P(A), where P(A) in the denominator is computed using the Total Probability Theorem. Bayes' work, edited and published by Richard Price in 1763, laid the foundation for an entire branch of statistics. Bayes never published his theorem himself — it was found among his papers after his death. His friend Richard Price recognized its importance, completed the work, and submitted it to the Royal Society two years after Bayes died.
Related Concepts
- Conditional Probability — The Total Probability Theorem is built directly from conditional probabilities P(A|Bᵢ); understanding P(A|B) = P(A∩B)/P(B) is essential before applying the theorem.
- Bayes' Theorem — Bayes' Theorem uses P(A) computed by the Total Probability Theorem in its denominator to invert conditional probabilities: P(Bᵢ|A) = P(A|Bᵢ)·P(Bᵢ)/P(A).
- Multiplication Rule of Probability — The step P(A∩Bᵢ) = P(A|Bᵢ)·P(Bᵢ) used in the derivation is exactly the Multiplication Rule; reviewing it solidifies the logical chain of the theorem.
- Sample Space and Events — A partition is defined over the sample space S; understanding what events, unions, and intersections are is necessary to grasp why the decomposition A = ∪(A∩Bᵢ) is valid.