Axioms of Probability

Kolmogorov's three lines are the load-bearing foundation of every quantitative science that reasons under uncertainty. In equilibrium statistical mechanics the sample space is phase space, $\mathcal{F}$ its Borel algebra, and the canonical ensemble is the measure $\mathrm{e}^{-\beta H}/Z$ — the partition function $Z$ is exactly the normalization constant enforcing axiom II, and every ensemble average is an integral against this measure. In quantum theory the Born rule assigns $P(k)=\lvert\langle k\mid\psi\rangle\rvert^2$, and Gleason's theorem (1957) shows that on a Hilbert space of dimension at least three the only measures consistent with the Kolmogorov axioms on the lattice of projectors take this quadratic form, with $\sum_k\lvert\langle k\mid\psi\rangle\rvert^2=1$ following from $\langle\psi\mid\psi\rangle=1$. The union bound of Derivation 3 is the daily workhorse of randomized algorithms and concentration inequalities, bounding the chance that any of many rare failures occurs with no knowledge of their dependence; the whole edifice of statistical inference and machine learning is written in the language of $(\Omega,\mathcal{F},P)$. The one genuinely open frontier is foundational: whether quantum probability is truly Kolmogorovian or requires a non-commutative generalization (quantum probability theory, contextuality) remains a live question at the border of physics and mathematics.

Picture the sample space $\Omega$ as a region and the probability measure $P$ as a fixed amount of paint — exactly one unit of it — spread across that region. The probability of an event $A$ is the amount of paint covering the subregion $A$. The three axioms are the three obvious rules of paint: you cannot have a negative amount (non-negativity), the whole region holds exactly one unit (normalization), and the paint on two non-overlapping patches adds (additivity). The entire elementary calculus of probability is the arithmetic of covered area. The first figure makes it concrete: two overlapping events, and the inclusion–exclusion bookkeeping that keeps the shared paint from being counted twice.

The single deep idea in Kolmogorov's axioms hides inside the word countable. Finite additivity — paint on finitely many disjoint patches adds — is a rule anyone would grant. Kolmogorov demanded more: additivity over a countably infinite family of disjoint events. That upgrade is exactly equivalent to a continuity property: as an increasing sequence of events $A_n$ swells to fill a limiting event $A$, their probabilities $P(A_n)$ climb continuously to $P(A)$ with no jump at the limit. This is the bridge between a static rule about area and the dynamic theorems physicists use — that relative frequencies converge and sample averages settle down. The second figure shows continuity on a nested family whose probabilities march to one.

The $\sigma$-algebra $\mathcal{F}$ is the least glamorous of the three objects, but it is what keeps the system honest: it is the collection of questions you are allowed to ask. On a die, every one of the $2^6$ subsets of $\{1,\dots,6\}$ is a fair question, so $\mathcal{F}$ is the full power set and nothing is lost. On the real line it is different: no countably additive measure can assign a sensible length to every subset, because pathological non-measurable sets (Vitali's construction) exist. The remedy is to ask only about the Borel sets — the $\sigma$-algebra generated by intervals — on which length, and hence probability, is well defined. Physicists almost never meet a non-measurable set, but the restriction is what guarantees the machinery never silently breaks.

The whole subject rests on one triple and three axioms; everything below is a specialization or a consequence. A probability space is a sample space of outcomes, a $\sigma$-algebra of measurable events, and a measure assigning each event a number in $[0,1]$.

where: $\Omega$ — sample space, the set of all possible outcomes; $\mathcal{F}$ — the $\sigma$-algebra of events, subsets of $\Omega$ closed under complement and countable union; $P$ — the probability measure, assigning a number in $[0,1]$ to each event. Reading: the triple is the complete specification of a random experiment — outcomes, admissible questions, and their likelihoods. The three axioms are the entire content of the theory: every theorem in elementary probability is an algebraic consequence of these three lines together with the closure properties of $\mathcal{F}$.

where: $A,A_i$ — events, i.e. elements of $\mathcal{F}$; in axiom III the $A_i$ are pairwise disjoint, $A_i\cap A_j=\varnothing$ for $i\neq j$. Reading: probabilities are non-negative, the certain event has probability one, and the probability of a countable disjoint union is the sum of the probabilities. Axiom III with a finite family (padded with copies of $\varnothing$) contains finite additivity as a special case. The event space cannot be arbitrary; it must be a $\sigma$-algebra, so operations on events never lead outside the domain of $P$. These three closure properties are what "$\sigma$-algebra" means.

where: $A^c=\Omega\setminus A$ — the complement of $A$; $\bigcup_i A_i$ — a countable union. Reading: closure under complement and countable union gives, by De Morgan, closure under countable intersection too — so every event built from countably many "and", "or", "not" operations is again measurable. The first fruits of the axioms are the empty-set value, the complement rule, and monotonicity — the rules you reach for constantly; all three are proved in Derivation 1.

where: $A\subseteq B$ — $A$ is a sub-event of $B$. Reading: the impossible event carries no probability, an event and its complement share the unit mass, and a larger event never carries less probability (monotonicity), which forces $0\le P(A)\le 1$ automatically. For events that overlap, additivity fails and must be replaced by inclusion–exclusion, which subtracts the double-counted intersection. This is the two-event case; the $n$-event alternating sum is Derivation 3.

where: $A\cap B$ — the joint event, subtracted once because it is counted in both $P(A)$ and $P(B)$. Reading: additivity is the special case $P(A\cap B)=0$; whenever events can co-occur, the overlap must be removed. The substantive consequence of countable additivity is continuity: probability commutes with monotone limits of events. This is the property that makes limit theorems provable, and it is proved in Derivation 2.

where: $A_n\uparrow A$ — an increasing sequence $A_1\subseteq A_2\subseteq\cdots$ with union $A=\bigcup_n A_n$; $A_n\downarrow A$ — a decreasing sequence with intersection $A=\bigcap_n A_n$. Reading: as events grow to fill a limit or shrink to one, their probabilities converge with no jump — the analytic backbone of the laws of large numbers. Finally, conditioning is not a fourth axiom but a definition built from the three: the probability of $A$ given that $B$ has occurred renormalizes the measure to live inside $B$.

where: $P(A\mid B)$ — the conditional probability of $A$ given $B$, itself a bona fide probability measure (it satisfies all three axioms with total mass renormalized to one on $B$). Reading: conditioning restricts the sample space to $B$ and rescales so its mass is again one — the launching point for Bayes' theorem.

The same three axioms specialize, without modification, to every concrete setting a physicist meets — the difference between "discrete" and "continuous" probability is entirely a difference in which measure $P$ one puts on $(\Omega,\mathcal{F})$, not a difference in the axioms.

1 · Two-event algebra from the axioms (confidence builder)

Two events satisfy $P(A)=0.6$, $P(B)=0.5$, and $P(A\cap B)=0.3$. Find (a) $P(A\cup B)$, (b) $P(A^c\cap B^c)$, (c) $P(A\cap B^c)$, and (d) the probability that exactly one of the two events occurs.

Step 1. Union by inclusion–exclusion, equation (5): $P(A\cup B)=0.6+0.5-0.3=\mathbf{0.8}$. The overlap $0.3$ is subtracted once so it is not counted in both $P(A)$ and $P(B)$.

Step 2. Neither event, via De Morgan and the complement rule: $A^c\cap B^c=(A\cup B)^c$, so $P(A^c\cap B^c)=1-0.8=\mathbf{0.2}$ — the outside region of the first figure, confirming the four disjoint pieces sum to one.

Step 3. $A$ but not $B$: split $A$ by the overlap, so $P(A\cap B^c)=0.6-0.3=\mathbf{0.3}$; by symmetry $P(B\cap A^c)=0.5-0.3=0.2$.

Step 4. Exactly one event is the disjoint union of the two "only" pieces: $P(\text{exactly one})=0.3+0.2=\mathbf{0.5}$, equivalently $P(A\cup B)-P(A\cap B)=0.8-0.3=0.5$. Every answer is a rearrangement of the three axioms — no new assumptions entered.

2 · Countable additivity in action on the integers

Let the sample space be the positive integers $\Omega=\{1,2,3,\dots\}$ with $P(\{n\})=2^{-n}$. Verify this is a valid probability measure, then compute (a) $P(\text{outcome is even})$, (b) $P(\text{outcome}\ge 3)$, and (c) $P(\text{outcome} \in\{1,2\})$ two different ways.

Step 1. Check normalization (axiom II) using countable additivity over the singletons: $P(\Omega)=\sum_{n=1}^\infty 2^{-n}=\dfrac{1/2}{1-1/2}=\mathbf{1}$, a convergent geometric series. Non-negativity is immediate since $2^{-n}>0$, so $P$ is a genuine probability measure on the infinite discrete space — a setting where finite additivity alone could not even state normalization.

Step 2. The outcome is even on the countable disjoint union over $n=2,4,6,\dots$: $P(\text{even})=\sum_{k=1}^\infty 2^{-2k}=\sum_{k=1}^\infty\left(\tfrac14\right)^k=\dfrac{1/4}{1-1/4}=\mathbf{\tfrac13}$. Axiom III is doing real work — "even" is an infinite disjoint union of singletons.

Step 3. Probability the outcome is at least $3$: fastest via the complement rule, $P(\{n\ge 3\})=1-P(\{1\})-P(\{2\})=1-\tfrac12-\tfrac14=\mathbf{\tfrac14}$, matching the direct sum $\sum_{n=3}^\infty 2^{-n}=2^{-2}=\tfrac14$.

Step 4. Probability the outcome is $1$ or $2$: by finite additivity $P(\{1,2\})=\tfrac12+\tfrac14=\mathbf{\tfrac34}$, agreeing with the complement of Step 3, $1-\tfrac14=\tfrac34$. On an infinite sample space, countable additivity is what makes normalization and "even/odd" probabilities well defined at all.

The axioms of probability are the work of one man who took a subject rich in results but poor in foundations and rebuilt it on measure theory — turning probability from a collection of clever tricks into a chapter of analysis.

Texts. Kolmogorov, Foundations of the Theory of Probability (1933; Chelsea trans. 1956) — the original three pages · Billingsley, Probability and Measure, 3rd ed., §§1–3 (the rigorous measure-theoretic development) · Stone & Goldbart, Mathematics for Physics, Ch. on probability and measure · Arfken, Weber & Harris, Mathematical Methods for Physicists, 7th ed., Ch. 23 · Riley, Hobson & Bence, Mathematical Methods for Physics and Engineering, 3rd ed., Ch. 30 · Feller, An Introduction to Probability Theory and Its Applications, Vol. 1 (peerless for intuition). The key results in this article — $P(\varnothing)=0$, $P(A^c)=1-P(A)$, monotonicity, two-event and $n$-event inclusion–exclusion, and the union bound — are all consequences of axioms I–III, as are the equivalence of countable additivity and continuity from below. The worked-example numbers ($P(A\cup B)=0.8$, $P(A^c\cap B^c)=0.2$, $P(\text{exactly one})=0.5$) and the geometric-measure values ($\sum 2^{-n}=1$, $P(\text{even})=1/3$, $P(n\ge 3)=1/4$) all follow directly from these axioms. All figures carry only the illustrative probabilities stated in their captions.

Prerequisites. Set theory and Boolean operations · Sequences, limits and series · Sigma-algebras and measure. Leads to. Conditional Probability and Bayes' Theorem · Random Variables and Probability Distributions · Laws of Large Numbers and Convergence of Random Variables · Central Limit Theorem.

Ask the tutor: Ask me to walk through any of the derivations line by line, or to build a probability space from scratch for a random experiment you have in mind — dice, radioactive decay, or a quantum measurement.