Conditional Probability and Bayes' Theorem

Conditional probability is the mathematics of learning. It answers one sharp question: once you know that an event $B$ has happened, how should your assessment of another event $A$ change? The answer is disarmingly simple — throw away every outcome inconsistent with $B$ , keep the slice that survives, and rescale it so the survivors again sum to one. That division by $P(B)$ is the whole of conditioning, and from it flow the multiplication rule, the chain rule, the law of total probability, and Bayes' theorem, together the computational spine of all probabilistic reasoning. Built directly on the Axioms of Probability, conditioning is not a new kind of object: for a fixed $B$ with $P(B)>0$ , the map $A\mapsto P(A\mid B)$ is itself a probability measure obeying every Kolmogorov axiom. That single fact — proved below — licenses everything that follows: any identity true of ordinary probabilities stays true inside the conditioned world. The crown of the subject is Bayes' theorem, the logical engine of inference. A physical model runs forward: it hands you the likelihood $P(D\mid H)$ of seeing data $D$ if hypothesis $H$ were true. But you want the inverse — the posterior $P(H\mid D)$ , how much to believe $H$ now that $D$ is in hand. Bayes' theorem performs that inversion, weighting the likelihood by the prior $P(H)$ and normalising by the total evidence $P(D)$ . Physicists meet it whenever they fit a model to noisy data, weigh a tentative detection against the background, or confront the base-rate trap in which a rare truth makes even an excellent test mostly cry wolf. The discrete story here passes seamlessly to continuous densities, the gateway to statistical data analysis.

Key takeaways

Conditional probability $P(A\mid B)=P(A\cap B)/P(B)$ measures how likely $A$ is once $B$ is known, by restricting the sample space to $B$ and renormalising.
The multiplication rule $P(A\cap B)=P(A\mid B)P(B)$ and its chain-rule extension decompose any joint probability into a product of sequential conditionals.
The law of total probability writes an unconditional probability as a prior-weighted average of likelihoods over a partition of exhaustive hypotheses.
Bayes' theorem inverts a conditional: $P(H\mid D)\propto P(D\mid H)\,P(H)$ — posterior equals likelihood times prior, normalised by the evidence.
Independence is the degenerate case $P(A\mid B)=P(A)$ , equivalently $P(A\cap B)=P(A)P(B)$ : $B$ carries no news about $A$ .
The odds form $\text{posterior odds}=\text{Bayes factor}\times\text{prior odds}$ turns inference into repeated multiplication, so sequential and batch updating agree.
Key idea: conditioning renormalises probability onto the known event; Bayes' theorem then reverses the direction of a conditional by weighting the likelihood with the prior and dividing by the total evidence.

The core insight

Conditioning renormalises probability onto the known event; Bayes' theorem then lets you reverse the direction of a conditional by weighting the likelihood with the prior and dividing by the total evidence.

Where this lives today

Bayes' theorem is the quiet arithmetic behind a striking amount of modern physics. In gravitational-wave astronomy, the LIGO–Virgo–KAGRA collaborations extract the masses, spins, and sky location of merging black holes by sampling the posterior $p(\theta\mid \text{strain data})$ over a fifteen-dimensional parameter space with a physical waveform likelihood — every published source measurement is a credible region of that posterior. In cosmology, the Planck satellite's determination of the six standard-model parameters, including the Hubble constant, is a Bayesian fit of the cosmic-microwave-background power spectrum quoted with posterior means and $68\%$ intervals. In particle physics the same logic weighs a candidate signal against background, and the base-rate warning above is why a $3\sigma$ excess is treated with caution and only a $5\sigma$ result is called a discovery — the diagnostic test writ large across the look-elsewhere effect. Sequential Bayesian updating powers the Kalman filters that track particles through detectors and spacecraft through orbit — all, at bottom, computing likelihood times prior over evidence.

The physical picture

Picture the sample space $\Omega$ as a tile of unit area, with events $A$ and $B$ as overlapping patches; the probability of $A$ is just its area. Now someone tells you $B$ has occurred. Everything outside $B$ is impossible, so you erase it; what remains of $A$ is only the overlap $A\cap B$ . But the survivors no longer add to one, so you magnify the picture by $1/P(B)$ until $B$ fills the frame, and the magnified overlap is exactly $P(A\mid B)=P(A\cap B)/P(B)$ . Conditioning is that two-step gesture — slice, then rescale. The first figure makes it concrete: an event $A$ with unconditional probability $0.20$ can have conditional probability $0.30$ once $B$ magnifies the overlap.

Figure

Conditioning slices and rescales: learning that B occurred discards everything outside B and magnifies the survivors so they sum to one, so P(A given B) = P(A∩B)/P(B) = 0.12/0.40 = 0.30 even though the unconditional P(A) = 0.20

Bayes' theorem is the same overlap counted twice. The joint $P(A\cap B)$ reads as "the fraction of $B$ that is also $A$ " or "the fraction of $A$ that is also $B$ "; equating the two readings and solving is the entire inversion — no deeper magic than reconciling a row-total with a column-total in a table. Inference then becomes iterated conditioning: the posterior after the first datum is the prior for the second, so one-at-a-time updates land exactly where a single batch update does. The most important — and most counter-intuitive — consequence is the base-rate effect, which the second figure makes unavoidable by counting people instead of manipulating fractions.

Figure

The base-rate punchline in natural frequencies: of 10 000 people at 1% prevalence, 100 are ill and 9 900 well; a test with 99% sensitivity and 95% specificity flags 99 true positives but 495 false positives, so a positive result means illness with probability 99/(99+495) = 16.7%

The core equations

Definition of conditional probability. Everything begins here. The conditional probability of $A$ given $B$ is the joint probability of both, divided by the probability of the event you now take as certain; the denominator is the rescaling that keeps the total at one, and it demands $P(B)>0$ .

P(A\mid B)=\frac{P(A\cap B)}{P(B)},\qquad P(B)>0 \qquad (1)

where: $P(A\mid B)$ — probability of $A$ once $B$ is known to have occurred; $P(A\cap B)$ — the joint probability that both occur; $P(B)$ — probability of the conditioning event, strictly positive. Read it as: of the world that is $B$ , what fraction is also $A$ .

Multiplication rule. Clearing the denominator in (1), with the roles of $A$ and $B$ also swapped, writes the joint probability two symmetric ways. This symmetry is the seed of Bayes' theorem.

P(A\cap B)=P(A\mid B)\,P(B)=P(B\mid A)\,P(A) \qquad (2)

where: $P(B\mid A)$ — probability of $B$ given $A$ . The joint is the probability of the first event times the conditional probability of the second — in either order.

Chain rule. Applying the multiplication rule repeatedly factors any joint probability of $n$ events into a product of conditionals, each conditioned on all its predecessors — the backbone of Markov chains, Bayesian networks, and sequential inference.

P(A_1\cap\cdots\cap A_n)=\prod_{k=1}^{n}P\!\left(A_k\,\middle|\,A_1\cap\cdots\cap A_{k-1}\right) \qquad (3)

where: $A_k$ — the $k$ -th event of an ordered sequence, with the $k=1$ factor read as the unconditional $P(A_1)$ . The order is free — every ordering gives the same product.

Law of total probability. If a set of hypotheses $\{H_i\}$ partitions the sample space — mutually exclusive and jointly exhaustive — then the probability of any event $B$ is the average of its conditional probabilities, weighted by the priors. It is how you compute a marginal from conditionals.

P(B)=\sum_{i} P(B\mid H_i)\,P(H_i) \qquad (4)

where: $\{H_i\}$ — a partition with $H_i\cap H_j=\varnothing$ for $i\neq j$ and $\sum_i P(H_i)=1$ . The unconditional probability is a prior-weighted mixture of likelihoods.

Bayes' theorem (discrete). Combining the symmetric multiplication rule (2) with the total-probability expansion (4) of the denominator gives the inversion formula. It converts likelihoods $P(D\mid H_i)$ , which a forward model supplies, into posteriors $P(H_i\mid D)$ , which inference demands.

P(H_i\mid D)=\frac{P(D\mid H_i)\,P(H_i)}{\sum_j P(D\mid H_j)\,P(H_j)} \qquad (5)

where: $P(H_i)$ — prior probability of hypothesis $H_i$ ; $P(D\mid H_i)$ — likelihood of data $D$ under $H_i$ ; $P(H_i\mid D)$ — posterior probability. Posterior $\propto$ likelihood $\times$ prior; the denominator is the evidence that makes the posteriors sum to one.

Bayes' theorem (continuous parameter inference). For a continuous parameter $\theta$ the same statement holds with probabilities replaced by densities and the sum by an integral — the workhorse of physical parameter estimation.

p(\theta\mid x)=\frac{p(x\mid\theta)\,p(\theta)}{\int p(x\mid\theta')\,p(\theta')\,\mathrm{d}\theta'} \qquad (6)

where: $\theta$ — the parameter to infer; $p(\theta)$ — prior density; $p(x\mid\theta)$ — likelihood, read as a function of $\theta$ at fixed data $x$ ; $p(\theta\mid x)$ — posterior density. The denominator, the marginal likelihood, is often the hard part of real inference.

Odds form and the Bayes factor. Divide (5) for two hypotheses to cancel the shared evidence. Inference becomes multiplication: the posterior odds are the prior odds times the likelihood ratio, and stacking independent data just multiplies the factors.

\frac{P(H_1\mid D)}{P(H_2\mid D)}=\underbrace{\frac{P(D\mid H_1)}{P(D\mid H_2)}}_{\text{Bayes factor}}\cdot\frac{P(H_1)}{P(H_2)} \qquad (7)

where: the Bayes factor is the likelihood ratio that updates prior odds to posterior odds. Because the evidence $P(D)$ cancels, ranking hypotheses never requires computing it.

Independence. Finally, the degenerate case in which conditioning changes nothing: $A$ and $B$ are independent exactly when their joint factorises, equivalently when $P(A\mid B)=P(A)$ .

A\perp\!\!\!\perp B \iff P(A\cap B)=P(A)\,P(B) \iff P(A\mid B)=P(A) \qquad (8)

where: $\perp\!\!\!\perp$ — statistical independence. Knowing $B$ then tells you nothing about $A$ ; the likelihood ratio in (7) collapses to one.

Derivation AConditioning is itself a probability measure

The single fact that makes the whole edifice safe is this: for a fixed conditioning event $B$ with $P(B)>0$ , the map $Q(\cdot)\equiv P(\cdot\mid B)$ is a genuine probability measure on the original sample space $\Omega$ . If $Q$ obeys the three Kolmogorov axioms — non-negativity, normalisation, and countable additivity — then every theorem ever proved about probabilities applies verbatim inside the conditioned world, which is what justifies iterating conditionals and treating conditional distributions as first-class objects. We verify the axioms one at a time using only the definition and the properties of the underlying measure $P$ .

Step 1. Non-negativity. Write $Q(A)=P(A\cap B)/P(B)$ . The numerator is a probability, hence $\ge 0$ , and the denominator is $P(B)>0$ by assumption, so $Q(A)\ge 0$ for every event $A$ . Why: a ratio of a non-negative quantity to a strictly positive one cannot be negative, so the first axiom transfers for free.

Step 2. Normalisation. Take $A=\Omega$ . Since $\Omega\cap B=B$ , we get $Q(\Omega)=P(\Omega\cap B)/P(B)=P(B)/P(B)=1$ . Why: intersecting the whole space with $B$ returns $B$ , so the rescaling by $1/P(B)$ is exactly what forces the total conditional probability to one — this is the renormalisation made rigorous.

Step 3. Countable additivity. Let $A_1,A_2,\dots$ be pairwise disjoint. Then the sets $A_k\cap B$ are also pairwise disjoint, and distributivity gives $\left(\bigcup_k A_k\right)\cap B=\bigcup_k(A_k\cap B)$ . Applying additivity of $P$ in the numerator, $Q\!\left(\bigcup_k A_k\right)=\frac{1}{P(B)}\sum_k P(A_k\cap B)=\sum_k Q(A_k)$ . Why: intersection with a fixed set preserves disjointness, so additivity passes straight through the division by the constant $P(B)$ .

Step 4. All three axioms hold, so $Q=P(\cdot\mid B)$ is a probability measure. Consequently every identity of probability — complementation $Q(A^c)=1-Q(A)$ , inclusion–exclusion, and Bayes' theorem itself — remains valid after replacing $P(\cdot)$ by $P(\cdot\mid B)$ . Why: conditioning does not leave the theory, it recreates the theory on a smaller world — the license for everything downstream.

Go deeper · Derivation B · 1 minBayes' theorem from the multiplication rule and total probability

Derivation BBayes' theorem from the multiplication rule and total probability

The goal is to derive the inversion formula (5) and its self-normalising denominator from nothing but the definition of conditional probability, and to see why the evidence is exactly what guarantees the posteriors sum to one. The strategy: write the single joint probability $P(H_i\cap D)$ in the two symmetric ways the multiplication rule permits, solve for the quantity we want, and expand the marginal $P(D)$ over a partition of hypotheses. Every line is elementary algebra applied to equation (1); the content is entirely in reading the same joint probability from two directions.

Step 1. Write the joint two ways. Conditioning on $D$ gives $P(H_i\cap D)=P(H_i\mid D)\,P(D)$ ; conditioning on $H_i$ gives $P(H_i\cap D)=P(D\mid H_i)\,P(H_i)$ . Equating them, $P(H_i\mid D)\,P(D)=P(D\mid H_i)\,P(H_i)$ . Why: the joint probability is one number, so the two factorisations must agree — the hinge on which the whole inversion turns.

Step 2. Solve for the posterior. Dividing by $P(D)>0$ , $P(H_i\mid D)=P(D\mid H_i)\,P(H_i)/P(D)$ . Why: this already is Bayes' theorem in unnormalised form — posterior proportional to likelihood times prior — with the denominator a single hypothesis-independent constant.

Step 3. Expand the evidence. Because $\{H_j\}$ partitions the space, $D=\bigcup_j(D\cap H_j)$ is a disjoint union, so $P(D)=\sum_j P(D\cap H_j)=\sum_j P(D\mid H_j)\,P(H_j)$ by the law of total probability. Why: the evidence is not an independent input — it is fully determined by the same likelihoods and priors, which is why Bayesian updating needs no information beyond the model.

Step 4. Substitute Step 3 into Step 2 to obtain equation (5). Summing the result over $i$ , the numerator sums to the denominator, so $\sum_i P(H_i\mid D)=1$ automatically. Why: the total-probability denominator is precisely the normaliser that turns a set of proportional weights into a proper posterior distribution. *The denominator identity guarantees $\sum_i P(H_i\mid D)=1$ exactly, confirming that the posteriors form a proper probability distribution.*

Go deeper · Derivation C · 1 minThe odds form and why sequential updating equals batch updating

Derivation CThe odds form and why sequential updating equals batch updating

Here the aim is to recast Bayes' theorem so the evidence disappears and inference becomes pure multiplication, and then to prove the practically vital fact that updating on a batch of independent data at once gives the identical posterior as updating one point at a time — what makes real-time filters and streaming analyses legitimate rather than approximate. We take the ratio of the theorem for two competing hypotheses to kill the shared denominator, then feed conditionally independent data through the chain rule and watch the likelihood factors accumulate.

Step 1. Form the odds. Writing (5) for $H_1$ and $H_2$ and dividing, the common evidence cancels: $\dfrac{P(H_1\mid D)}{P(H_2\mid D)}=\dfrac{P(D\mid H_1)}{P(D\mid H_2)}\cdot\dfrac{P(H_1)}{P(H_2)}$ , i.e. posterior odds $=$ Bayes factor $\times$ prior odds. Why: ratios sidestep the marginal likelihood, so comparing hypotheses is always cheaper than scoring one in isolation.

Step 2. Add a second, conditionally independent datum. If $P(D_1\cap D_2\mid H)=P(D_1\mid H)\,P(D_2\mid H)$ for each hypothesis, the joint Bayes factor factorises: $\dfrac{P(D_1\cap D_2\mid H_1)}{P(D_1\cap D_2\mid H_2)}=\dfrac{P(D_1\mid H_1)}{P(D_1\mid H_2)}\cdot\dfrac{P(D_2\mid H_1)}{P(D_2\mid H_2)}$ . Why: conditional independence turns the evidence of a data set into a product of per-datum factors, so evidence accumulates multiplicatively.

Step 3. Update in either order. Batch: multiply the prior odds by both factors at once. Sequential: multiply by the first factor, adopt the result as the new prior, then multiply by the second. Multiplication is associative and commutative, so both routes reach the same final odds. Why: the posterior after datum one legitimately becomes the prior for datum two — the chain rule in action, and what licenses recursive estimators like the Kalman filter.

Step 4. Take logarithms: $\log(\text{posterior odds})=\log(\text{prior odds})+\sum_k \log(\text{Bayes factor}_k)$ , so evidence adds linearly in log-odds. Why: independent measurements contribute additively in log-odds, which is why detection statistics are built from summed log-likelihood ratios. *The two-datum sequential update reproduces the batch posterior exactly — as worked numerically in Example 2, where 16.7% updates to 79.8% by two applications of the same likelihood ratio 19.8.*

Limiting cases

Bayes' theorem and the conditional definition collapse to familiar statements in special regimes, each a useful sanity check:

Case	Result	Meaning
Independence	$P(A\mid B)=P(A)$	$B$ carries no information about $A$ ; equivalently $P(A\cap B)=P(A)P(B)$ , and the Bayes factor is $1$ .
Certain evidence	$P(H\mid D)=1$ when $P(D\mid H')=0$ for all $H'\neq H$	if only one hypothesis can produce $D$ , observing $D$ proves it, provided its prior was nonzero.
Flat prior	$P(H\mid D)\propto P(D\mid H)$	with equal priors the posterior tracks the likelihood — Bayesian inference reduces to maximum likelihood.
Zero prior	$P(H\mid D)=0$ whenever $P(H)=0$	no amount of data can resuscitate a hypothesis assigned prior zero (Cromwell's rule).
Overwhelming data	$P(H\mid D_{1:n})\xrightarrow{n\to\infty}\delta_{H,H_\star}$	as independent data accumulate the posterior concentrates on the true hypothesis, swamping any nonzero prior.

Common trapConfusing P(A given B) with P(B given A) — the prosecutor's fallacy

The two conditionals are not interchangeable; they are related by Bayes' theorem, $P(A\mid B)=P(B\mid A)\,P(A)/P(B)$ , and coincide only in the accident $P(A)=P(B)$ . The classic failure is to hear "the probability of this evidence if the defendant were innocent is one in a million" and conclude "the probability the defendant is innocent given the evidence is one in a million" — ignoring that the second requires the prior odds of guilt, which the first says nothing about. The same slip, in physics, is reading a small p-value $P(\text{data}\mid \text{no signal})$ as if it were the posterior $P(\text{no signal}\mid \text{data})$ . When the prior for new physics is tiny, a 3σ excess can still be far more likely a fluctuation than a discovery. Always carry the base rate $P(H)$ ; a likelihood without a prior cannot be turned into a posterior.

Worked examples

1 · The base-rate trap in a diagnostic test (fully worked)

A disease has prevalence $1\%$ . A screening test has sensitivity $99\%$ (it catches $99\%$ of true cases) and specificity $95\%$ (it correctly clears $95\%$ of the healthy). A randomly screened person tests positive. What is the probability they actually have the disease? Let $D$ denote disease and $T^{+}$ a positive test. The given data are $P(D)=0.01$ , $P(T^{+}\mid D)=0.99$ , and, since specificity is $0.95$ , the false-positive rate $P(T^{+}\mid D^{c})=0.05$ .

Step 1. True positives — the prior-weighted likelihood for the diseased branch: $P(T^{+}\mid D)\,P(D)=0.99\times0.01=\mathbf{0.0099}$ , the fraction of the whole population that is both ill and correctly flagged.

Step 2. False positives — the healthy branch, where the population $P(D^{c})=0.99$ is enormous: $P(T^{+}\mid D^{c})\,P(D^{c})=0.05\times0.99=\mathbf{0.0495}$ . Five percent of a huge healthy majority is five times the true-positive count.

Step 3. Evidence — the total probability of a positive, summing both branches (law of total probability): $P(T^{+})=0.0099+0.0495=\mathbf{0.0594}$ .

Step 4. Posterior — Bayes' theorem is now just the diseased branch divided by the total: $P(D\mid T^{+})=0.0099/0.0594=1/6\approx\mathbf{16.7\%}$ . Despite a test that is $99\%$ sensitive, a positive result is wrong five times out of six, purely because the disease is rare. The posterior is 16.7%, not 99% — the false positives from the large healthy population swamp the true positives. Sensitivity to the prior: at $2\%$ prevalence the same arithmetic gives $\dfrac{0.99\times0.02}{0.99\times0.02+0.05\times0.98}=\dfrac{0.0198}{0.0688}\approx\mathbf{28.8\%}$ — doubling the prior nearly doubles the posterior, since in this regime the base rate dominates, not the test quality.

2 · Sequential updating with the odds form

The person from Example 1, having tested positive once (posterior $16.7\%$ ), takes a second independent test of the same quality and again tests positive. What is the probability of disease now?

Step 1. Convert to odds. The prior odds of disease before any test are $P(D)/P(D^{c})=0.01/0.99=1/99\approx0.0101$ . The likelihood ratio (Bayes factor) of a single positive is $\dfrac{P(T^{+}\mid D)}{P(T^{+}\mid D^{c})}=\dfrac{0.99}{0.05}=\mathbf{19.8}$ .

Step 2. First positive. Posterior odds $=$ prior odds $\times$ Bayes factor $=0.0101\times19.8=\mathbf{0.20}$ , giving posterior probability $0.20/1.20=1/6\approx16.7\%$ — reproducing Example 1, as it must.

Step 3. Second positive. Because the tests are conditionally independent, just multiply by the same factor again: posterior odds $=0.20\times19.8=\mathbf{3.96}$ .

Step 4. Convert back: $P(D\mid T^{+}T^{+})=3.96/(1+3.96)=3.96/4.96\approx\mathbf{79.8\%}$ . Two positives from a modest-prevalence screen push belief from a $1\%$ prior to nearly $80\%$ — and updating the two results one at a time gave the identical answer as multiplying both factors at once, exactly as Derivation 3 promised. Two independent positives raise the posterior from 16.7% to 79.8%; evidence multiplies in odds, adds in log-odds.

Physics of the portrait

Bayes' theorem is named for a Presbyterian minister who never published it, rescued from his papers by a friend, and turned into a science by a French astronomer who arrived at it independently and used it to weigh the mass of Saturn.

Origins · 1 minThomas Bayes and how this was discovered

Thomas Bayes

c. 1701–1761 · inverse probability and the doctrine of chances

Thomas Bayes was an English Presbyterian minister and Fellow of the Royal Society whose single, posthumous contribution reshaped how science reasons from evidence. In an unpublished manuscript, "An Essay towards solving a Problem in the Doctrine of Chances," Bayes asked the inverse question his contemporaries had avoided: given the successes and failures observed, what can be said about the underlying probability that produced them? Where Bernoulli and de Moivre had computed the probability of data given a known cause, Bayes inverted the arrow — from data back to cause. He died in 1761 with the essay unpublished; his friend Richard Price edited it and read it to the Royal Society in 1763. The real engine of the field, though, was Pierre-Simon Laplace, who from 1774 independently formulated the "probability of causes," generalised it far beyond Bayes' special case, and applied it to astronomy and geodesy — including estimating planetary masses from imperfect observations. For a century the method was simply "inverse probability." It fell out of favour among frequentists early in the twentieth century, then was rebuilt on firmer foundations by Harold Jeffreys, by R. T. Cox — whose 1946 theorem shows that any consistent measure of plausibility must obey the probability axioms, making Bayes' rule a logical necessity rather than a convention — and by Andrey Kolmogorov, whose 1933 axioms made conditional probability a rigorous measure. Edwin Jaynes later championed the resulting view of "probability as extended logic," and the modern renaissance owes as much to computation as to theory — but the skeleton is still the one line Price found among a dead minister's papers.

History

1763

— Price communicates Bayes' Essay on the doctrine of chances to the Royal Society

1774

— Laplace independently states the principle of inverse probability

1812

— Laplace's Théorie analytique des probabilités systematises the method

1933

— Kolmogorov's axioms make conditional probability a rigorous measure

1946

— Cox's theorem derives probability as the unique logic of consistent plausibility

Going further

Texts. Stone and Goldbart, Mathematics for Physics (CUP, 2009), probability chapter · Arfken, Weber and Harris, Mathematical Methods for Physicists, 7th ed., probability chapter · Riley, Hobson and Bence, Mathematical Methods for Physics and Engineering, 3rd ed., ch. 30 · Jaynes, Probability Theory: The Logic of Science (CUP, 2003) · Sivia and Skilling, Data Analysis: A Bayesian Tutorial, 2nd ed. · Gregory, Bayesian Logical Data Analysis for the Physical Sciences. Key numerical checks: the diagnostic-test arithmetic gives $0.0099/0.0594=1/6\approx0.167$ ; the $2\%$ prevalence variant gives $\approx0.288$ ; the two-positive sequential update gives $3.96/4.96\approx0.798$ via likelihood ratio $0.99/0.05=19.8$ ; and the independence check gives $\tfrac12\cdot\tfrac12=\tfrac14$ . The prevalence, sensitivity, and specificity are illustrative textbook values, not measured constants.

Prerequisites. Axioms of Probability · Random Variables and Probability Distributions. Leads to. Joint, Marginal and Conditional Distributions · Multivariate Gaussian Distribution · Laws of Large Numbers and Convergence of Random Variables.

Ask the tutor: Bring me any Bayesian update — your own priors and likelihoods, or a puzzle like Monty Hall or a rare-disease screen — and we will turn the crank together.

What is the difference between P(A given B) and P(B given A)?

They answer opposite questions and can differ enormously. $P(A\mid B)$ asks how likely $A$ is within the world where $B$ holds; $P(B\mid A)$ asks how likely $B$ is within the world where $A$ holds. Bayes' theorem is the bridge: $P(A\mid B)=P(B\mid A)\,P(A)/P(B)$ , so they are equal only when the priors $P(A)$ and $P(B)$ match. Swapping them silently is the prosecutor's fallacy — the probability that a test is positive given disease (high) is a completely different number from the probability of disease given a positive test (which, as the worked example shows, can be low when the disease is rare).

Why is the posterior only 17% when the test is 99% accurate?

Because "99% accurate" describes the test's behaviour on known cases — the likelihood — while the question asks about a person from a population that is $99\%$ healthy, which brings in the prior. Out of $10\,000$ people at $1\%$ prevalence, $100$ are ill and the test flags $99$ of them, but $9\,900$ are healthy and it wrongly flags $5\%$ , i.e. $495$ people. So $594$ positives arise and only $99$ are real: $99/594\approx17\%$ . The false positives from the huge healthy majority outnumber the true positives from the tiny sick minority — the base-rate effect, correct accounting rather than a paradox.

Where does the prior come from, and isn't it subjective?

The prior encodes what you knew before the data — a measured prevalence, a previous experiment's posterior, a physical constraint like a mass being positive, or a deliberately vague "flat" distribution. Two analysts with different priors can reach different posteriors from the same data, which critics call subjective. Two replies temper the worry: with enough data the likelihood dominates and posteriors from any reasonable prior converge, and the prior is stated openly and can be challenged, unlike the hidden assumptions in many "objective" procedures. In physics the prior is often uncontroversial — parameters have known ranges, and sensitivity to it is reported.

What does the law of total probability actually do for me?

It lets you compute the probability of an event by splitting the world into cases you can handle. If a set of hypotheses covers every possibility without overlap, then $P(B)=\sum_i P(B\mid H_i)P(H_i)$ assembles the answer as a prior-weighted average of conditional probabilities. Operationally it fills in the denominator of Bayes' theorem — the evidence $P(D)$ is just the law of total probability applied over the hypothesis partition. It is also the everyday tool for "condition on what you don't know, then average it out," which is how marginal distributions are computed throughout statistical physics.

Is independence the same as mutual exclusivity?

No — they are almost opposites, and confusing them is a common error. Two events are mutually exclusive if they cannot both happen, so $P(A\cap B)=0$ ; knowing one occurred tells you the other did not — a very strong dependence. Two events are independent if $P(A\cap B)=P(A)P(B)$ , meaning one occurring leaves the other's probability untouched. Mutually exclusive events with nonzero probabilities are necessarily dependent, since $P(A\mid B)=0\neq P(A)$ . The only way to be both is for one event to have probability zero.

How does the continuous version relate to the discrete Bayes' theorem?

It is the same statement with densities in place of point probabilities and an integral in place of the sum. For a continuous parameter $\theta$ , $p(\theta\mid x)=p(x\mid\theta)\,p(\theta)/\int p(x\mid\theta')\,p(\theta')\,\mathrm{d}\theta'$ : the posterior density is the likelihood times the prior density, normalised by the marginal likelihood. Everything conceptual carries over, but the normalising integral is often intractable, which is why modern Bayesian inference leans on sampling methods like Markov-chain Monte Carlo and nested sampling to explore the posterior without evaluating it directly.