Properties of Conditional Probability
Conditional probability measures the likelihood of an event occurring given that another event has already occurred. When we write P(A | B), we are asking: if we already know B happened, what is the probability that A also happened? This idea is central to reasoning under uncertainty and appears throughout statistics, machine learning, medicine, and everyday decision-making. The properties of conditional probability form a structured set of rules that allow us to manipulate and compute probabilities systematically. These rules — including the multiplication rule, the law of total probability, and Bayes' theorem — all flow from a single definition, making conditional probability one of the most elegant and powerful structures in mathematics. Understanding these properties lets you solve complex problems by breaking them into manageable parts. Whether you are analyzing medical test accuracy, predicting weather, or building recommendation algorithms, the rules governing conditional probability provide the mathematical foundation you need.
- The Definition
- Property 1 — It Is Still a Valid Probability
- Property 2 — The Multiplication Rule
- Property 3 — The Law of Total Probability
- Property 4 — Bayes' Theorem
- Property 5 — Independence as a Special Case
The Definition
Conditional probability is defined as P(A | B) = P(A ∩ B) / P(B), provided P(B) > 0. This formula restricts the sample space to only the outcomes inside B, then measures what fraction of those also lie inside A. Everything else follows from this one equation.
Property 1 — It Is Still a Valid Probability
Conditional probability satisfies all three Kolmogorov axioms: (1) P(A | B) ≥ 0 for any event A. (2) P(B | B) = 1, meaning the conditioning event itself is certain given it occurred. (3) If A₁ and A₂ are mutually exclusive, then P(A₁ ∪ A₂ | B) = P(A₁ | B) + P(A₂ | B). This means every theorem that applies to ordinary probability also applies to conditional probability.
Property 2 — The Multiplication Rule
Rearranging the definition gives the multiplication rule: P(A ∩ B) = P(A | B) · P(B). This can be extended to three events: P(A ∩ B ∩ C) = P(A | B ∩ C) · P(B | C) · P(C). The multiplication rule is essential for computing joint probabilities when direct observation is difficult.
Property 3 — The Law of Total Probability
If {B₁, B₂, …, Bₙ} is a partition of the sample space (mutually exclusive, collectively exhaustive events), then for any event A: P(A) = Σᵢ P(A | Bᵢ) · P(Bᵢ). This law lets you compute an overall probability by conditioning on each possible 'scenario' Bᵢ separately, then combining the results using the weights P(Bᵢ).
Property 4 — Bayes' Theorem
Combining the definition with the law of total probability yields Bayes' theorem: P(Bᵢ | A) = [P(A | Bᵢ) · P(Bᵢ)] / [Σⱼ P(A | Bⱼ) · P(Bⱼ)]. This theorem allows you to 'reverse' the conditioning — to update a prior belief P(Bᵢ) into a posterior belief P(Bᵢ | A) after observing evidence A.
Property 5 — Independence as a Special Case
Events A and B are independent if and only if P(A | B) = P(A), which is equivalent to P(A ∩ B) = P(A) · P(B). Independence means that knowing B happened gives you no new information about A. This is a special case, not the default — most real-world events are dependent.
The Key Equation
- P(A \mid B) — Conditional probability (dimensionless): The probability that event A occurs, given that event B has occurred.
- P(A \cap B) — Joint probability (dimensionless): The probability that both events A and B occur simultaneously.
- P(B) — Marginal probability of B (dimensionless): The probability that event B occurs; must be strictly greater than zero.
Worked Example
Problem. A bag contains 4 red and 6 blue marbles. Two marbles are drawn without replacement. (a) Find the probability that the second marble is red given the first was red. (b) Use the law of total probability to find P(second marble is red).
Answer. P(R₂ | R₁) = 1/3 and P(R₂) = 2/5
Origins · 1 minThomas Bayes and how this was discovered
Thomas Bayes, an English statistician and Presbyterian minister, is credited with formulating the theorem that now bears his name. His essay 'An Essay towards solving a Problem in the Doctrine of Chances' was published posthumously in 1763 by his friend Richard Price. The essay introduced a method for computing inverse probabilities — what we today call posterior probabilities — and laid the foundation for Bayesian inference, one of the two dominant schools of statistics. Bayes never published his theorem himself. His friend Richard Price found the manuscript among his papers after his death and submitted it to the Royal Society. Bayes may not have even considered it significant enough to publish during his lifetime.
Related Concepts
- Bayes' Theorem — Bayes' theorem is the most powerful consequence of the definition of conditional probability, allowing beliefs to be updated with new evidence.
- Independence of Events — Independence is defined precisely through conditional probability: A and B are independent when P(A|B) = P(A).
- Law of Total Probability — This law is a direct application of conditional probability properties and is essential for computing marginal probabilities from conditional ones.
- Sample Spaces and Events — Understanding what events are and how they relate is a prerequisite for making sense of intersections like A ∩ B in the conditional probability formula.