Total probability and Bayes' theorem

A factory floor with three machines, or an urn room with several bags, poses the same question in disguise: an event you actually observe — a defective item, a red ball — could have arisen through more than one "route." The Theorem of Total Probability answers the forward question, how likely is the event overall, once you know how likely each route is and how likely the event is along each route. Bayes' Theorem answers the reverse question that examiners love: given that the event did happen, how likely is it that a particular route was responsible? This is the same conditional-probability and multiplication-theorem machinery you already know, applied systematically to a sample space that has been split into non-overlapping pieces. Get the partition right and both formulas become almost mechanical.

The core insight

When a sample space splits into a partition $\{E_i\}$ , the overall probability of an event $A$ is $P(A)=\sum_i P(E_i)P(A|E_i)$ , and Bayes' Theorem inverts this sum to give the revised, "after-the-fact" probability $P(E_i|A)$ of each cause.

The core idea

A partition splits the sample space cleanly. Events $E_1, E_2, \dots, E_n$ form a partition of the sample space $S$ if they are mutually exclusive ( $E_i \cap E_j = \varnothing$ for $i \ne j$ ) and exhaustive ( $E_1 \cup E_2 \cup \cdots \cup E_n = S$ ) — every outcome belongs to exactly one $E_i$ . Think of $E_i$ as "which machine produced the item" or "which bag was picked," and let $A$ be an event whose chance depends on which $E_i$ occurred, such as "the item is defective."

Total probability weights each route by how likely it is. Since $A$ can be written as the disjoint union $A = (A\cap E_1)\cup(A\cap E_2)\cup\cdots\cup(A\cap E_n)$ , the probabilities add, and each piece expands by the multiplication theorem:

P(A) = \sum_{i=1}^{n} P(E_i)\,P(A\mid E_i)

where: $P(E_i)$ is the prior probability of route $i$ (known before $A$ is observed), and $P(A\mid E_i)$ is the conditional probability that $A$ occurs given route $i$ .

Bayes' Theorem reverses the question. Once $A$ is known to have happened, the revised probability that it came from route $E_i$ is

P(E_i \mid A) = \frac{P(E_i)\,P(A\mid E_i)}{\displaystyle\sum_{j=1}^{n} P(E_j)\,P(A\mid E_j)}

where: the numerator is the single term for route $i$ , and the denominator is exactly $P(A)$ from the total-probability sum — so Bayes' Theorem is just $P(E_i\mid A) = P(E_i\cap A)/P(A)$ with $P(A)$ expanded over the partition. The output $P(E_i\mid A)$ is called the posterior probability.

Derivation ATotal probability sums the routes, Bayes divides out one of them

The goal is to show, starting from nothing but the definition of conditional probability and the fact that a partition splits an event into disjoint pieces, exactly why $P(A) = \sum_i P(E_i)P(A\mid E_i)$ holds and why Bayes' Theorem then follows immediately as a ratio using that same sum, so that no separate proof is ever needed for the denominator.

Step 1. Split $A$ using the partition. Since $E_1,\dots,E_n$ cover $S$ with no overlap, intersecting each $E_i$ with $A$ slices $A$ itself into disjoint pieces: $A = (A\cap E_1)\cup(A\cap E_2)\cup\cdots\cup(A\cap E_n)$ . No outcome of $A$ is double-counted or missed, because every outcome lies in exactly one $E_i$ .

Step 2. Add the disjoint pieces. Disjoint events have additive probability, so $P(A) = P(A\cap E_1)+P(A\cap E_2)+\cdots+P(A\cap E_n) = \sum_i P(A\cap E_i)$ .

Step 3. Expand each piece with the multiplication theorem. Since $P(A\cap E_i) = P(E_i)\,P(A\mid E_i)$ , substituting gives $P(A) = \sum_i P(E_i)\,P(A\mid E_i)$ — the Theorem of Total Probability.

Step 4. Reverse it for Bayes' Theorem. By definition, $P(E_i\mid A) = \dfrac{P(E_i\cap A)}{P(A)} = \dfrac{P(E_i)\,P(A\mid E_i)}{P(A)}$ . Replacing the denominator $P(A)$ with the Step 3 sum gives $P(E_i\mid A) = \dfrac{P(E_i)P(A\mid E_i)}{\sum_j P(E_j)P(A\mid E_j)}$ , so the same total-probability calculation supplies both the answer to "how likely is $A$ overall" and the denominator needed to reverse the question for any one route.

Worked example

A factory has three machines, I, II, and III, producing 25%, 35%, and 40% of the day's output, with defective rates 5%, 4%, and 2% respectively. An item is picked at random and found to be defective; find (a) the overall probability an item is defective, and (b) the probability it came from machine II.

Step 1. Set up the partition. Let $E_1, E_2, E_3$ be "item is from machine I, II, III," with $P(E_1)=0.25$ , $P(E_2)=0.35$ , $P(E_3)=0.40$ — mutually exclusive and exhaustive, so this is a valid partition. Let $A$ = "item is defective," with $P(A\mid E_1)=0.05$ , $P(A\mid E_2)=0.04$ , $P(A\mid E_3)=0.02$ .

Step 2. Apply the Theorem of Total Probability for part (a).

P(A) = (0.25)(0.05)+(0.35)(0.04)+(0.40)(0.02) = 0.0125+0.0140+0.0080 = 0.0345

So $3.45\%$ of the day's output is defective overall.

Step 3. Apply Bayes' Theorem for part (b). The numerator is the machine-II term already computed, $P(E_2)P(A\mid E_2)=0.0140$ , and the denominator is $P(A)=0.0345$ from Step 2 — no fresh calculation needed.

P(E_2\mid A) = \frac{0.0140}{0.0345} = \frac{140}{345} = \frac{28}{69} \approx 0.4058

Answer: $P(A) = 0.0345$ , and $P(E_2\mid A) \approx 0.4058$ , so about $40.6\%$ of defective items came from machine II — higher than machine II's raw $35\%$ output share, because its $4\%$ defect rate sits above the blended average, while machine III (biggest output, lowest defect rate) contributes proportionally less to the defective pool.

Competitive stretch

Using the same data, find $P(E_1\mid A)$ and $P(E_3\mid A)$ , and check they add up correctly with $P(E_2\mid A)$ .

Step 1. Reuse the weighted terms and the denominator from the main example: $P(E_1)P(A\mid E_1)=0.0125$ , $P(E_2)P(A\mid E_2)=0.0140$ , $P(E_3)P(A\mid E_3)=0.0080$ , with $P(A)=0.0345$ .

Step 2. Divide each weighted term by $P(A)$ : $P(E_1\mid A) = 0.0125/0.0345 \approx 0.3623$ , and $P(E_3\mid A) = 0.0080/0.0345 \approx 0.2319$ .

Step 3. Check the sum: $0.3623 + 0.4058 + 0.2319 \approx 1.0000$ . This must always hold — the posteriors $P(E_i\mid A)$ over a full partition add to exactly $1$ , since a defective item must have come from some machine; it is a fast way to catch an arithmetic slip in a Bayes' Theorem question with several routes.

Common traps

Applying either theorem without first checking $\{E_i\}$ is a genuine partition (mutually exclusive and exhaustive) — the formulas simply do not hold otherwise.
Quoting the prior $P(E_i)$ as if it were the final Bayes' answer, instead of computing the ratio $P(E_i)P(A\mid E_i)/P(A)$ .
Recomputing the Bayes' denominator from scratch rather than reusing the total-probability sum already found — wastes time and invites arithmetic slips.
Adding the conditional probabilities $P(A\mid E_i)$ directly instead of weighting each by its prior $P(E_i)$ first; every term in the sum must be a product, not $P(A\mid E_i)$ alone.

Ask the tutor: if you would like help identifying the partition $\{E_i\}$ in a trickier word problem, or want a second Bayes'-Theorem example worked through with fresh numbers, ask the AI tutor for a guided walkthrough.

Why is P(A) in the Bayes denominator not just P(A) computed some other way?

It is the same $P(A)$ you would get by any correct method, but the fastest route is to reuse the Theorem of Total Probability sum, since $A$ has already been split over the partition $\{E_i\}$ for the numerator. Recomputing it independently is extra work and a common source of arithmetic mismatches between numerator and denominator.

What is the real difference between a prior and a posterior probability here?

The prior $P(E_i)$ is your belief about which route occurred before you know whether $A$ happened — for example, machine II's 35% output share. The posterior $P(E_i\mid A)$ is the revised belief after learning $A$ occurred, such as the item being defective. Bayes' Theorem is precisely the rule for updating the prior into the posterior using the evidence.

Do I always need three or more events in the partition, or can Bayes theorem work with just two?

Two events work fine as long as they are mutually exclusive and exhaustive, such as $E_1$ = "coin is fair" and $E_2$ = "coin is biased." Exam problems favour three or more routes (three machines, three bags) because it exercises the summation more thoroughly, but the algebra of both theorems is identical for any $n \ge 2$ .

Total probability and Bayes' theorem

The core idea

Worked example

Competitive stretch

Common traps

Before this

Where this leads