Probability Spaces — Sigma-Algebras and Kolmogorov Axioms

A probability space is a triple $(Ω, F, P)$ where $Ω$ is a sample space, $F$ a $σ$ -algebra of measurable events, and $P$ a countably additive measure with $P (Ω) = 1$ . The Kolmogorov axioms (1933) are: non-negativity, normalization, and countable additivity. This is the rigorous measure-theoretic foundation of all modern probability, statistics, stochastic processes, and mathematical finance. Due to Andrey Kolmogorov, Grundbegriffe der Wahrscheinlichkeitsrechnung, 1933.

A probability space formalizes the notion of a random experiment. The sample space $Ω$ contains all possible outcomes; the $σ$ -algebra $F \subseteq 2^{Ω}$ specifies precisely which subsets of outcomes (events) can be assigned a probability; and the probability measure $P : F \to [0, 1]$ assigns consistent likelihoods obeying countable additivity. This structure matters because naive set-theoretic probability collapses when $Ω$ is uncountable — one cannot consistently assign probabilities to all subsets of $[0, 1]$ (the Vitali construction produces non-measurable sets). The surprising fact is that probability theory is, technically, just measure theory with the single normalizing constraint $P (Ω) = 1$ , yet this constraint plus independence generates the entire distinctive theory of random variables, conditioning, and limit theorems.

The Core Insight

The genius of the Kolmogorov framework is separating "what can be measured" from "how it is measured." The $σ$ -algebra answers the former (it is a closure-under-countable-operations structure), and the measure answers the latter. Countable additivity — not mere finite additivity — is the precise amount of "continuity" needed to make limits, infinite sequences, and convergence theorems work, while still being weak enough to be realizable. Everything in probability is downstream of this one design choice.

Formal Statement

Sigma-Algebra

Let $Ω$ be a nonempty set. A collection $F \subseteq 2^{Ω}$ is a $σ$ -algebra (or $σ$ -field) on $Ω$ if:

Ω \in F

⟳The whole space is always an event — it is the certain event with probability 1.

A \in F ⟹ A^{c} = Ω ∖ A \in F

⟳Closure under complementation; "not A" is an event whenever "A" is.

A_{1}, A_{2}, A_{3}, \dots \in F ⟹ n = 1 ⋃ \infty A_{n} \in F

⟳Closure under countable unions; the word "

σ

" denotes "countable."

Here $Ω$ is the sample space, elements $ω \in Ω$ are outcomes, and members $A \in F$ are events. From [1]–[3], $F$ is automatically closed under countable intersections (via De Morgan) and contains $\emptyset$ .

Probability Measure and the Kolmogorov Axioms

Given a measurable space $(Ω, F)$ , a function $P : F \to R$ is a probability measure if it satisfies the three Kolmogorov axioms:

P (A) \geq 0 for all A \in F (non-negativity)

⟳Probabilities measure "amount of belief/frequency" and cannot be negative.

P (Ω) = 1 (normalization)

⟳The certain event has total mass 1; this is the only axiom distinguishing probability from general measure theory.

A_{i} \cap A_{j} = \emptyset (i \neq = j) ⟹ P (n = 1 ⋃ \infty A_{n}) = n = 1 \sum \infty P (A_{n}) (countable additivity)

⟳Disjoint events' probabilities add; the countable (not just finite) version is what enables limit theorems.

The triple $(Ω, F, P)$ is then a probability space.

Elementary Consequences of the Axioms

From [1]–[6] the following hold for all $A, B \in F$ :

P (\emptyset) = 0, P (A^{c}) = 1 - P (A), 0 \leq P (A) \leq 1

⟳Apply additivity to

Ω = A ⊔ A^{c}

and use

P (Ω) = 1

A \subseteq B ⟹ P (A) \leq P (B) (monotonicity)

⟳Write

B = A ⊔ (B ∖ A)

and apply non-negativity.

P (A \cup B) = P (A) + P (B) - P (A \cap B) (inclusion-exclusion)

⟳Decompose

A \cup B

into three disjoint pieces and add.

P (n = 1 ⋃ \infty A_{n}) \leq n = 1 \sum \infty P (A_{n}) (countable subadditivity / union bound)

⟳Disjointify via

B_{n} = A_{n} ∖ ⋃_{k < n} A_{k}

, apply additivity then monotonicity.

Continuity of Probability Measures

Countable additivity is equivalent (given finite additivity) to either form of sequential continuity. For $A_{n} ↑ A$ (increasing, $A = ⋃ A_{n}$ ) and $B_{n} ↓ B$ (decreasing, $B = ⋂ B_{n}$ ):

A_{n} ↑ A ⟹ P (A_{n}) ↑ P (A); B_{n} ↓ B ⟹ P (B_{n}) ↓ P (B)

⟳For the increasing case, write

A

as a disjoint union of telescoping rings

A_{n} ∖ A_{n - 1}

and apply countable additivity.

Why It Holds / Derivation

Equivalence of Countable Additivity and Continuity from Above at $\varnothing$

We show: a finitely additive $P$ with $P (Ω) = 1$ is countably additive iff for every decreasing sequence $C_{n} ↓ \emptyset$ we have $P (C_{n}) \to 0$ .

Step 1 (Additivity $\Rightarrow$ continuity). Let

C_{n} ↓ \emptyset

. Define disjoint rings

D_{n} = C_{n} ∖ C_{n + 1}

, so

C_{1} = ⨆_{n \geq 1} D_{n}

. By countable additivity

P (C_{1}) = \sum_{n \geq 1} P (D_{n})

, a convergent series. The tail satisfies

P (C_{m}) = \sum_{n \geq m} P (D_{n}) \to 0

m \to \infty

since the series converges. Step 2 (Continuity $\Rightarrow$ additivity). Let

{A_{n}}

be disjoint with union

A = ⨆_{n} A_{n}

. Put

C_{m} = A ∖ ⨆_{n = 1}^{m} A_{n} = ⨆_{n > m} A_{n}

. Then

C_{m} ↓ \emptyset

, so

P (C_{m}) \to 0

. By finite additivity,

P (A) = n = 1 \sum m P (A_{n}) + P (C_{m}) .

Step 3. Letting

m \to \infty

and using

P (C_{m}) \to 0

gives

P (A) = \sum_{n = 1}^{\infty} P (A_{n})

■

Necessity of a Sigma-Algebra: Why Not All Subsets?

Claim. No countably additive, translation-invariant

P

exists on all of

2^{[0, 1)}

with

P ([0, 1)) = 1

. Step 1. Define

x \sim y

iff

x - y \in Q

. By the axiom of choice, pick one representative from each equivalence class, forming the Vitali set

V \subseteq [0, 1)

. Step 2. Enumerate

Q \cap [0, 1) = {q_{1}, q_{2}, \dots}

and let

V_{k} = (V + q_{k}) mod 1

. These are pairwise disjoint and

⨆_{k} V_{k} = [0, 1)

. Step 3. By translation invariance all

P (V_{k})

are equal to some constant

c

. Countable additivity gives

1 = \sum_{k = 1}^{\infty} c

, which is

0

c = 0

and

\infty

c > 0

. Contradiction. Conclusion. Hence

V

cannot be assigned a consistent probability; one must restrict to a

σ

-algebra (e.g. the Borel or Lebesgue

σ

-algebra) that excludes such pathological sets.

■

Andrey Nikolaevich Kolmogorov | 1903–1987

Kolmogorov, working in Moscow, unified probability with Lebesgue's measure theory in his 1933 monograph Grundbegriffe der Wahrscheinlichkeitsrechnung ("Foundations of the Theory of Probability"). Before him, probability was a loose collection of combinatorial rules and intuitions plagued by paradoxes (Bertrand's paradox, ill-defined "random points"). Kolmogorov's insight was that a random experiment is exactly a normalized measure space, and that conditional probability and independence are additional structures layered on top. He went on to found vast areas — turbulence theory, algorithmic complexity (Kolmogorov complexity), and the theory of stochastic processes (the Kolmogorov extension theorem, forward/backward equations).

Building Intuition

Think of $Ω$ as the universe of "everything that could happen," and $P$ as one unit of sand spread over it. An event $A$ is a region; $P (A)$ is the amount of sand sitting on that region. The $σ$ -algebra $F$ is the catalog of regions whose sand-content is well-defined — some regions (Vitali sets) are too jagged to weigh consistently, so we exclude them.

Countable additivity says: if you partition a region into infinitely many disjoint sub-regions, the sand adds up exactly — no sand is "lost in the limit." This is precisely the property that fails for mere finite additivity, and it is the lifeblood of every convergence theorem: the law of large numbers, the central limit theorem, and martingale convergence all secretly depend on being able to take limits inside $P$ .

Architecture of a probability space: outcomes, events, and the measure

Worked Examples

Finite Sample Space — A Fair Die

Given. Roll a fair six-sided die.

Ω = {1, 2, 3, 4, 5, 6}

F = 2^{Ω}

(all

2^{6} = 64

subsets), and

P ({ω}) = \frac{1}{6}

for each

ω

. Verify the axioms. Non-negativity [4] holds since each singleton has

P = \frac{1}{6} > 0

. Normalization [5]:

P (Ω) = \sum_{k = 1}^{6} \frac{1}{6} = 1

. Additivity [6] holds by construction since

Ω

is finite (countable additivity reduces to finite additivity here). Compute. Let

A = {

even

} = {2, 4, 6}

and

B = {> 3} = {4, 5, 6}

$P (A) = \frac{3}{6} = \frac{1}{2}$ , $P (B) = \frac{3}{6} = \frac{1}{2}$ .
$A \cap B = {4, 6}$ , so $P (A \cap B) = \frac{2}{6} = \frac{1}{3}$ .
By inclusion–exclusion [9]: $P (A \cup B) = \frac{1}{2} + \frac{1}{2} - \frac{1}{3} = \frac{2}{3}$ .

Check directly: $A \cup B = {2, 4, 5, 6}$ , four outcomes, $P = \frac{4}{6} = \frac{2}{3}$ . ✓

P (A \cup B) = \frac{2}{3} \approx 0.667 (dimensionless)

:::example[Uncountable Sample Space — Uniform on $[0, 1]$ ] Given. Pick a point "uniformly at random" in $[0, 1]$ . Take $Ω = [0, 1]$ , $F = B ([0, 1])$ the Borel $σ$ -algebra, and $P = λ$ the Lebesgue measure restricted to $[0, 1]$ , so $P ([a, b]) = b - a$ .

Verify normalization [5].

P ([0, 1]) = 1 - 0 = 1

. ✓ A key subtlety — singletons have measure zero. For any single point

x

P ({x}) = n \to \infty lim P ([x, x + \frac{1}{n}] \cap [0, 1]) = n \to \infty lim \frac{1}{n} = 0,

using continuity from above [11]. Thus every individual outcome has probability 0, yet

P (Ω) = 1

. This is impossible for finite sample spaces and is the hallmark of continuous probability. Compute a concrete event. Let

A = [0.2, 0.5] \cup [0.7, 0.9]

P (A) = (0.5 - 0.2) + (0.9 - 0.7) = 0.3 + 0.2 = 0.5.

P (A) = 0.5 (dimensionless)

Note that a countable union of singletons, e.g. $Q \cap [0, 1]$ , has $P = \sum_{q} 0 = 0$ by countable additivity [6] — the rationals are "negligible," whereas the irrationals carry full probability 1.

Common Misconceptions

Finite additivity is NOT enough. Many introductory treatments quietly assume "probabilities of disjoint events add," but only specify finite additivity. There exist genuinely different finitely-additive measures (e.g. those built from free ultrafilters / Banach limits) that are not countably additive, and for which the continuity property [11] and the limit theorems fail. Kolmogorov deliberately demanded countable additivity [6]. Conversely, do not over-claim: there is no consistent way to demand uncountable additivity — that would force

P ([0, 1]) = 0

from

\sum_{x} P ({x}) = \sum_{x} 0

. "Event" $\neq =$ "outcome," and " $P (A) = 0$ " $\neq =$ " $A$ is impossible." In a continuous space the event "the dart hits exactly

x = 0.5

" has probability 0 but is not impossible — it can occur. Likewise

P (A) = 1

does not mean

A = Ω

; it means

A

holds almost surely. The empty event

\emptyset

is the only "truly impossible" event in the logical sense, but null events behave like it for measure purposes.

Non-measurable sets break naive probability

Suppose, contrary to the framework, that we insisted $F = 2^{[0, 1)}$ and that $P$ were translation-invariant and countably additive with $P ([0, 1)) = 1$ . The Vitali set $V$ (proof above) yields $⨆_{k} (V + q_{k}) mod 1 = [0, 1)$ with all pieces of equal probability $c$ , forcing $1 = \sum_{k = 1}^{\infty} c$ — impossible for any value of $c$ . Hence $V$ is non-measurable, and the $σ$ -algebra must be smaller than the full power set. This is not a technicality: it is the reason $σ$ -algebras exist.

Connections

The probability space is the launching point for the whole edifice. A Random Variable is a measurable function $X : (Ω, F) \to (R, B)$ , and its Probability Distribution is the pushforward measure $P \circ X^{- 1}$ on $R$ . Conditional Probability and Conditional Expectation are defined relative to sub- $σ$ -algebras $G \subseteq F$ , which encode "information." Independence (Probability) is a property of how $σ$ -algebras factorize. The entire apparatus of Measure Theory — Carathéodory extension, Dynkin's $π$ – $λ$ theorem, the monotone class theorem — supplies the existence and uniqueness machinery that guarantees these objects are well defined. The Kolmogorov Extension Theorem then lets us build infinite-dimensional spaces hosting Stochastic Processes.

Real-World Applications

Mathematical finance — Arbitrage pricing requires an equivalent martingale measure $Q$ on the path space of asset prices; the Black–Scholes model lives on a Wiener probability space $(Ω, F, P)$ with filtration.
Statistical inference — Likelihood, sufficiency, and the Neyman–Pearson lemma are all statements about Radon–Nikodym derivatives between probability measures on a common measurable space.
Quantum mechanics — Outcome statistics of measurements form genuine Kolmogorov probability spaces per observable, though the family across non-commuting observables famously cannot be glued into one (Bell/Kochen–Specker).
Information theory — Shannon entropy $H (X) = - \sum p (x) lo g p (x)$ is a functional of a probability measure; channel capacity is an optimization over input distributions.
Machine learning — PAC learning bounds, generalization error, and Bayesian posteriors are all phrased over probability spaces of data and hypotheses; concentration inequalities use the union bound [10].
Reliability engineering — System failure probabilities combine component events via inclusion–exclusion [9] and fault-tree subadditivity.
Statistical mechanics — The microcanonical/canonical ensembles are probability measures on phase space $Ω$ ; observables are integrals against these measures.

Historical Development

1654 — Pascal–Fermat correspondence on the problem of points launches combinatorial probability.
1713 — Jacob Bernoulli's Ars Conjectandi states the (weak) law of large numbers for Bernoulli trials.
1812 — Laplace's Théorie analytique des probabilités codifies the classical "equally likely cases" definition.
1889 — Bertrand's paradox exposes the ambiguity of "random" without a specified measure.
1902–1904 — Lebesgue and Borel develop measure and integration theory, supplying the missing rigorous backbone.
1909 — Borel's strong law of large numbers and "normal numbers" first use measure-theoretic arguments in probability.
1923 — Wiener constructs the rigorous measure (Wiener measure) for Brownian motion on path space.
1933 — Kolmogorov's Grundbegriffe axiomatizes probability as measure theory, introducing the triple $(Ω, F, P)$ , conditional expectation via $σ$ -algebras, and the extension theorem.
1930s–40s — Doob develops martingale theory and the modern theory of stochastic processes atop the Kolmogorov foundation.

Subtleties & Limits

Foundational tensions remain at the edges. (i) Finitely-additive probability (de Finetti, Dubins–Savage) is a coherent alternative championed for subjective probability; whether countable additivity is "the right" axiom is philosophically contested, since it rules out a uniform distribution on

N

. (ii) Non-measurable phenomena depend on the axiom of choice: under the axiom of determinacy (AD), every subset of

R

is Lebesgue measurable, so the Vitali pathology disappears — but AD contradicts full AC. (iii) Quantum probability does not fit a single Kolmogorov space across incompatible observables, motivating non-commutative (

C^{*}

-algebraic) generalizations where states replace measures. (iv) Existence of large product/extension measures on uncountable index sets can require careful regularity hypotheses (Polish/standard Borel spaces) for the Kolmogorov extension theorem to apply; on arbitrary spaces consistency of finite-dimensional marginals need not yield a process.