Conditional Probability

Conditional probability is the probability of an event under the assumption that some other event has already happened, effectively shrinking the sample space to only the outcomes consistent with the given condition. It matters because almost all real-world reasoning involves updating beliefs as evidence arrives — raw, unconditional probabilities rarely capture what we actually know. The defining quantity $P(A\mid B)$ is the proportion of the "world where $B$ is true" in which $A$ is also true. A surprising fact is that $P(A\mid B)$ and $P(B\mid A)$ can be wildly different — confusing the two (the "prosecutor's fallacy") has led to real miscarriages of justice. From this single definition flow the multiplication rule, Bayes' theorem, and the entire concept of statistical independence.

The Core Insight

When you learn that $B$ occurred, you discard every outcome where $B$ is false and renormalize what remains so the surviving probabilities again sum to 1. Conditional probability is nothing more than restricting attention to the sub-universe of outcomes consistent with your evidence, then asking how much of that sub-universe also satisfies $A$. The division by $P(B)$ is precisely this renormalization.

Formal Statement

Conditional Probability

Let $A$ and $B$ be events in a sample space $\Omega$ with a probability measure $P$, and suppose $P(B)>0$. The conditional probability of $A$ given $B$ is:

Here:

• $\Omega$ = the sample space (set of all possible outcomes),
• $A,B\subseteq \Omega$ = events,
• $A\cap B$ = the event that both $A$ and $B$ occur,
• $P(B)$ = the (unconditional) probability of $B$, required to be strictly positive,
• $P(A\mid B)$ = probability of $A$ given that $B$ has occurred.

Rearranging equation [1] gives the multiplication rule, often more useful in practice:

Bayes' Theorem

For events $A$ and $B$ with $P(A)>0$ and $P(B)>0$:

When the denominator must be expanded over a partition $\{A_1,A_2,\dots ,A_n\}$ of $\Omega$ (the law of total probability):

Independence

Events $A$ and $B$ are independent if and only if:

Why It Holds / Derivation

Justifying the Definition via Renormalization

We show that equation [1] is the unique sensible definition for "probability of $A$ within the world where $B$ holds."

1. Restrict the universe. Given that $B$ occurred, only outcomes inside $B$ remain possible. We seek a new probability measure $P_B$ defined on subsets of $B$.

1. Proportionality. Within $B$, relative likelihoods should be preserved — if outcome $x$ was twice as likely as $y$ before, it still is. So $P_B(A)=c\cdot P(A\cap B)$ for some constant $c$, since only the part of $A$ inside $B$ matters.

1. Normalization. A valid probability requires $P_B(B)=1$. Thus $1=c\cdot P(B\cap B)=c\cdot P(B)$, giving $c=1/P(B)$.

1. Conclude. Therefore $P_B(A)=\frac{P(A\cap B)}{P(B)}$, which is exactly $P(A\mid B)$ in equation [1]. $■$

That Conditional Probability Is Itself a Valid Probability

Fix $B$ with $P(B)>0$. We verify $P(\cdot \mid B)$ satisfies the Kolmogorov axioms.

1. Non-negativity: $P(A\mid B)=P(A\cap B)/P(B)\ge 0$ since both factors are non-negative.

1. Total mass: $P(\Omega \mid B)=P(\Omega \cap B)/P(B)=P(B)/P(B)=1$.

1. Countable additivity: For disjoint $A_1,A_2,\dots$, the sets $A_i\cap B$ are also disjoint, so

Hence $P(\cdot \mid B)$ is a genuine probability measure. $■$

Thomas Bayes | c. 1701–1761

An English Presbyterian minister and amateur mathematician, Bayes left an unpublished essay solving the "inverse probability" problem: given observed outcomes, how do we infer the probability of an underlying cause? His friend Richard Price edited and published it posthumously in 1763 as An Essay towards solving a Problem in the Doctrine of Chances. Bayes considered a thought experiment of balls rolled on a table — reasoning backward from where later balls land to where an unseen first ball lies. Though he never wrote the theorem in its modern form, the principle of reversing conditionals — turning $P(B\mid A)$ into $P(A\mid B)$ — bears his name and underlies an entire school of statistical inference.

Building Intuition

Picture the sample space as a rectangle of total area 1. Event $B$ is a region inside it; event $A$ is another region, possibly overlapping. The unconditional $P(A)$ is the fraction of the whole rectangle covered by $A$. But $P(A\mid B)$ throws away everything outside $B$ and asks: of the area now remaining (just $B$), what fraction is also $A$? That fraction is the overlap $A\cap B$ divided by all of $B$.

This is why conditioning can raise or lower a probability. If $A$ overlaps $B$ heavily, learning $B$ makes $A$ more likely. If they barely overlap, $B$ makes $A$ less likely. And if $A$'s share of $B$ equals its share of the whole space, then $B$ is irrelevant — that is independence.

Conditional probability as renormalization: restricting the universe to B

Worked Examples

Two Dice — The Sum Given a Six

Note that unconditionally $P(A)=5/36\approx 0.139$, so learning "at least one six" raised the probability of an 8.

Medical Test — Bayes in Action

Despite a 99% sensitive test, a positive result means only a ~17% chance of disease, because the disease is rare. This is the famous base-rate effect.

Common Misconceptions

Do Not Confuse $P(A \mid B)$ with $P(B \mid A)$

These are generally not equal. In the medical example, $P(+\mid D)=0.99$ but $P(D\mid +)\approx 0.17$. Swapping them is the prosecutor's fallacy: "the chance of this DNA match if innocent is 1 in a million, therefore the chance the defendant is innocent is 1 in a million" is invalid reasoning that ignores the base rate of innocent people. Equation [3] is precisely the correct way to convert one into the other.

Conditioning on a Zero-Probability Event

Equation [1] requires $P(B)>0$. If $P(B)=0$, the ratio $0/0$ is undefined and $P(A\mid B)$ has no meaning from the elementary definition. Handling such cases (e.g., conditioning on a continuous variable taking an exact value) requires the more advanced machinery of conditional densities and measure theory.

Independence Is Not "Mutual Exclusivity"

A common error is thinking disjoint (mutually exclusive) events are independent. They are nearly the opposite. If $A$ and $B$ are disjoint with positive probabilities, then $A\cap B=\varnothing$, so $P(A\mid B)=0\ne P(A)$. Knowing $B$ occurred tells you $A$ cannot have — that is maximal dependence, not independence. For example, "rolling a 2" and "rolling a 5" on one die are mutually exclusive but strongly dependent.

Connections

Conditional probability is the hinge connecting many topics. It generalizes Joint Probability via the multiplication rule [2], and it specializes to ordinary probability when the condition is $\Omega$. It defines Statistical Independence (equation [5]) and powers Bayes' Theorem for inference. The Law of Total Probability (equation [4]) decomposes events over a partition, and conditional probability underlies Conditional Expectation, Markov Chains (where the future depends only on the present state), and Random Variables through conditional distributions. It is also the foundation of Decision Theory under uncertainty.

Real-World Applications

1. Medical diagnosis (Bayesian screening): computing the true probability of disease given a positive test, correcting for base rates — as in the worked example above.
2. Spam filtering (Naive Bayes classifiers): $P(\text{spam}\mid \text{words})$ is computed from $P(\text{words}\mid \text{spam})$; this powered early Gmail and SpamAssassin filters.
3. Forensic science / DNA evidence: courts must convert match probabilities $P(\text{match}\mid \text{innocent})$ into guilt probabilities, where the prosecutor's fallacy is a documented danger (R v. Sally Clark, 1999).
4. Weather forecasting: "70% chance of rain given current pressure and humidity" is an inherently conditional statement updated as data arrives.
5. Insurance and actuarial pricing: premiums depend on $P(\text{claim}\mid \text{age, history, location})$.
6. Machine learning: classifiers, hidden Markov models, and Bayesian networks all encode conditional dependencies; the entire field of probabilistic inference rests on it.
7. Gambling and game theory: updating odds in poker or blackjack as cards are revealed is live conditional-probability calculation.

Historical Development

• 1654 — Blaise Pascal and Pierre de Fermat correspond about the "problem of points," laying groundwork for probability and implicitly conditional reasoning.
• 1713 — Jacob Bernoulli's Ars Conjectandi establishes the law of large numbers, framing probability as quantified uncertainty.
• 1763 — Thomas Bayes' essay is published posthumously by Richard Price, introducing inverse probability.
• 1774 & 1812 — Pierre-Simon Laplace independently develops and greatly extends Bayesian ideas in Théorie analytique des probabilités, giving the rule its general form.
• 1837 — Siméon Poisson clarifies conditional reasoning in his work on judgments and probability.
• 1933 — Andrey Kolmogorov's Grundbegriffe der Wahrscheinlichkeitsrechnung axiomatizes probability, defining conditional probability rigorously and extending it to general (measure-theoretic) conditioning.
• 1950s — Conditional probability becomes central to information theory (Shannon) and statistical decision theory (Wald).
• 1980s–1990s — Judea Pearl develops Bayesian networks, making conditional dependence a computational engine for AI.
• 2000s–present — Bayesian methods dominate machine learning, genomics, and data science.

Subtleties & Limits

Conditioning on events of probability zero — for example, "given that a randomly chosen point lies exactly on a line" — cannot be handled by equation [1] and leads to the Borel–Kolmogorov paradox, where the answer depends on how one approaches the limiting condition. The measure-theoretic resolution defines conditional expectation as a Radon–Nikodym derivative, but it is only specified "almost everywhere," leaving genuine philosophical ambiguity about conditioning on individual zero-probability outcomes. Foundational debates between frequentist and Bayesian interpretations of what $P(A\mid B)$ means — a long-run frequency versus a degree of belief — remain unresolved and actively shape statistical practice today.

See Also