The Axioms of Probability

The axioms underwrite every quantitative use of probability in modern science and engineering. In statistics, hypothesis testing and confidence intervals are built on probability measures satisfying exactly these axioms, whether the underlying space is discrete (counts of successes) or continuous (measurement error). In finance, option pricing models (e.g. the Black–Scholes framework) work with probability measures over continuous paths of stock prices, requiring the full strength of countable additivity to justify limiting arguments over infinite time partitions. In computer science and machine learning, probabilistic graphical models, Bayesian inference, and randomized algorithms all rest on axiomatic probability to guarantee that computed quantities are internally consistent — e.g. that posterior probabilities over an exhaustive, mutually exclusive set of hypotheses sum to one. In physics, statistical mechanics assigns probability measures to microstates of a system, and the Kolmogorov axioms guarantee that entropy and macroscopic averages are well-defined regardless of how one partitions the state space. In cryptography, the security proofs of protocols rely on precise probability bounds over the randomness used by an adversary, and inclusion-exclusion is a standard tool for bounding the probability of at least one of several "bad events" occurring during an attack. Finally, in probability theory itself, this axiomatic base is the springboard for Conditional Probability, where probabilities are re-normalized given partial information, and for Combinatorial Probability, where counting techniques compute $P(A)$ explicitly on finite equally-likely sample spaces consistent with these same axioms.

Ask the tutor: How does conditional probability, defined via $P(A\mid B) = P(A\cap B)/P(B)$, itself define a new probability measure satisfying all three Kolmogorov axioms?

Think of $P$ as a way of distributing one unit of "mass" over the sample space, with events being regions that collect mass. Axiom 1 says no region can have negative mass — mass is a nonnegative quantity, like area or weight. Axiom 2 says the whole space carries exactly one unit of mass in total: probability is a relative, normalized notion, not an absolute count. Axiom 3 is the subtle one: it says that if you chop a region into countably many non-overlapping pieces, the mass of the whole equals the sum of the masses of the pieces. This is exactly the property that area, length, and volume have — probability is a measure in the same technical sense used in analysis, restricted to total mass one. The reason additivity is demanded for countably infinite disjoint unions, not merely finite ones, is that many natural events (like "the first head appears on some finite toss," a countable union over "head first appears on toss $n$") are built from infinitely many disjoint pieces, and consistency across finite and infinite decompositions is essential for calculus with limits — this becomes crucial once you build continuous distributions later.

Example 1: A loaded three-sided spinner and inclusion-exclusion with three events. A spinner has three regions labeled $1, 2, 3$, but it is unfair: $P(\{1\}) = 0.5$, $P(\{2\}) = 0.3$, $P(\{3\}) = 0.2$. Two independent spins are made; let $\Omega = \{1,2,3\}^2$ with 9 outcomes $(i,j)$, and $P((i,j)) = P(\{i\}) \cdot P(\{j\})$ (independence is assumed for this construction, though the axioms alone don't require this — it's simply how we're defining $P$ here). Check normalization first:

confirming Axiom 2 holds for this $P$. Nonnegativity (Axiom 1) is clear since all factors are nonnegative products. Countable additivity (Axiom 3) holds automatically for any function defined this way on a finite sample space with $\mathcal{F} =$ all subsets, because finite unions of singletons behave additively by construction, and there are no genuinely infinite disjoint unions to worry about beyond finite ones (any countable union of nonempty disjoint subsets of a 9-point set must have only finitely many nonempty terms — the rest are forced to be $\emptyset$, contributing $0$ by part (a) above). Define three events on this 9-point space:

• $A$ = "first spin is 1" $= \{(1,j): j \in \{1,2,3\}\}$, so $P(A) = 0.5$.
• $B$ = "second spin is 1" $= \{(i,1): i \in \{1,2,3\}\}$, so $P(B) = 0.5$.
• $C$ = "the two spins match" $= \{(1,1),(2,2),(3,3)\}$, so $P(C) = 0.5^2+0.3^2+0.2^2 = 0.25+0.09+0.04=0.38$.

We compute $P(A \cup B \cup C)$ two ways to check inclusion–exclusion (part (g) with $n=3$). Pairwise intersections: $A \cap B = \{(1,1)\}$, so $P(A\cap B) = 0.5 \times 0.5 = 0.25$. $A \cap C = \{(1,1)\}$ (matching and first is 1 forces $(1,1)$), so $P(A \cap C) = 0.25$. $B \cap C = \{(1,1)\}$ similarly, so $P(B \cap C) = 0.25$. Triple intersection: $A \cap B \cap C = \{(1,1)\}$, so $P(A\cap B\cap C) = 0.25$. Inclusion–exclusion gives:

Direct check by listing the complement. The complement event is "first spin ≠ 1, second spin ≠ 1, and they don't match," i.e. $(i,j)$ with $i,j \in \{2,3\}$ and $i \ne j$: that's $(2,3)$ and $(3,2)$.

So $P((A\cup B\cup C)^c) = 0.06+0.06 = 0.12$, and by part (c) (complement rule),

which matches the inclusion–exclusion computation exactly, confirming the formula.

Example 2: Verifying monotonicity and the addition law with an asymmetric infinite sample space. Let $\Omega = \{1, 2, 3, \dots\}$ (all positive integers), $\mathcal{F}$ the collection of all subsets of $\Omega$, and define

extended to any event $A \subseteq \Omega$ by $P(A) = \sum_{n \in A} \tfrac{1}{2^n}$ (this sum always converges since it's bounded by the full series). This models, e.g., "number of tosses of a biased coin until first head," with heads probability $1/2$. Check Axiom 2: $P(\Omega) = \sum_{n=1}^\infty \frac{1}{2^n} = 1$ (geometric series with ratio $1/2$), confirming normalization. Check Axiom 3 with a genuine countably infinite disjoint union. Let $A_k = \{2k-1, 2k\}$ for $k=1,2,3,\dots$ (pairs: $\{1,2\}, \{3,4\}, \{5,6\},\dots$), which partition $\Omega$ into disjoint two-element blocks. Then

Countable additivity demands

Indeed $\bigcup_k A_k = \Omega$ and we already found $P(\Omega)=1$ directly — consistent. Now take $A = \{1,3,5,7,\dots\}$ (odd numbers, first head on an odd-numbered toss) and $B = \{1,2,3\}$ (first head within the first three tosses). Compute:

By the addition law (part (f)):

Direct check: $A \cup B = \{1,2,3,5,7,9,11,\dots\} = \{2\} \cup \{1,3,5,7,\dots\}$, i.e., $\{2\}$ together with all odd numbers. So $P(A\cup B) = P(\{2\}) + P(A) = \frac14 + \frac23 = \frac{3}{12}+\frac{8}{12} = \frac{11}{12}$, matching exactly. Finally, since $\{1,3\} \subset A$, monotonicity (part (d)) predicts $P(\{1,3\}) \le P(A)$: indeed $\frac58 = 0.625 \le \frac23 \approx 0.667$. ✓.