Probability — Sample Spaces, Events, and Axioms

Probability theory begins with a triple $(Ω, F, P)$ : a sample space of outcomes, a

σ

-algebra of events, and a probability measure satisfying three axioms (non-negativity, normalization, countable additivity). Every legitimate probabilistic statement is a theorem derivable from these axioms. Used everywhere quantitative uncertainty appears — statistics, physics, finance, machine learning, information theory. Axiomatized by Andrey Kolmogorov in 1933.

A sample space $Ω$ is the set of all possible outcomes of a random experiment; an event is a (measurable) subset of $Ω$ ; and a probability measure $P$ assigns to each event a number in $[0, 1]$ obeying Kolmogorov's axioms. This framework matters because it replaces vague intuitions ("the coin is fair") with a rigorous calculus in which probabilities of complex events are computed, not merely guessed. The deep idea is that probability is a special case of measure theory: $P$ is just a measure with total mass $1$ . A surprising fact is that not every subset of a sample space can consistently be assigned a probability — for uncountable $Ω$ such as $[0, 1]$ , the existence of non-measurable sets forces us to restrict events to a $σ$ -algebra rather than the full power set. This restriction is not a technicality but a structural necessity.

The Core Insight

Probability is not about outcomes but about sets of outcomes. Once you accept that events form a $σ$ -algebra (a collection closed under complement and countable union) and that $P$ distributes additively over disjoint events, all of elementary probability follows as deductive consequences. The axioms are deliberately minimal: three short statements generate the entire edifice — the inclusion–exclusion principle, conditional probability, Bayes' theorem, the law of total probability, and continuity of measure.

Formal Statement

Sample Space, Outcomes, Events

The sample space $Ω$ is the non-empty set of all possible outcomes of an experiment. Each element $ω \in Ω$ is an outcome (or elementary event, or sample point).

An event is a subset $A \subseteq Ω$ that belongs to a designated collection $F$ of "measurable" subsets. We say event $A$ occurs if the realized outcome $ω$ satisfies $ω \in A$ .

Special events:

$Ω$ — the certain event (always occurs).
$\emptyset$ — the impossible event (never occurs).
$A^{c} = Ω ∖ A$ — the complement (" $A$ does not occur").
$A \cup B$ — " $A$ or $B$ "; $A \cap B$ — " $A$ and $B$ ".
$A, B$ are mutually exclusive (disjoint) iff $A \cap B = \emptyset$ .

$\sigma$-algebra of Events

A collection $F$ of subsets of $Ω$ is a $σ$ -algebra (or $σ$ -field) if:

Ω \in F

⟳the certain event must be measurable; this anchors normalization.

A \in F ⟹ A^{c} \in F

⟳closure under complement — if "A occurs" is an event, so is "A does not occur".

A_{1}, A_{2}, A_{3}, \dots \in F ⟹ i = 1 ⋃ \infty A_{i} \in F

⟳closure under countable union; by De Morgan, combined with [2] this gives closure under countable intersection.

The pair $(Ω, F)$ is called a measurable space. Elements of $F$ are exactly the events.

Probability Measure — Kolmogorov's Axioms

A function $P : F \to R$ is a probability measure if it satisfies the three Kolmogorov axioms:

Axiom I (Non-negativity).

P (A) \geq 0 for all A \in F

⟳probabilities are sizes of sets; sizes are non-negative.

Axiom II (Normalization).

P (Ω) = 1

⟳the certain event has full measure; total mass is normalized to one.

Axiom III (Countable Additivity / $σ$ -additivity). For any sequence of pairwise disjoint events

A_{1}, A_{2}, \dots

(i.e.

A_{i} \cap A_{j} = \emptyset

for

i \neq = j

P (i = 1 ⋃ \infty A_{i}) = i = 1 \sum \infty P (A_{i})

⟳the probability of a disjoint union is the sum of the parts — additivity extended to countably many pieces, which is what enables limit/continuity arguments.

The triple $(Ω, F, P)$ is a probability space.

Elementary Consequences of the Axioms

From axioms [4]–[6] alone, for all events $A, B \in F$ :

P (\emptyset) = 0

⟳

Ω = Ω \cup \emptyset \cup \emptyset \cup \dots

disjointly; apply [6] and [5].

P (A^{c}) = 1 - P (A)

⟳

A \cup A^{c} = Ω

disjointly; use [6] then [5].

0 \leq P (A) \leq 1

⟳combine [4] with [8] (since

P (A^{c}) \geq 0

A \subseteq B ⟹ P (A) \leq P (B) (monotonicity)

⟳write

B = A \cup (B ∖ A)

disjointly; the extra term is non-negative.

P (A \cup B) = P (A) + P (B) - P (A \cap B) (inclusion-exclusion)

⟳decompose

A \cup B

into three disjoint pieces

A ∖ B

A \cap B

B ∖ A

and recombine.

P (i = 1 ⋃ \infty A_{i}) \leq i = 1 \sum \infty P (A_{i}) (countable subadditivity / union bound)

⟳disjointify via

B_{1} = A_{1}

B_{k} = A_{k} ∖ ⋃_{j < k} A_{j}

; then

B_{k} \subseteq A_{k}

and apply [6] and [10].

Proof of Finite Additivity and Inclusion–Exclusion

Step 1 — Finite additivity from [6]. Let

A_{1}, A_{2}

be disjoint. Extend to an infinite sequence

A_{1}, A_{2}, \emptyset, \emptyset, \dots

, which remains pairwise disjoint. By [6] and [7]:

P (A_{1} \cup A_{2}) = P (A_{1}) + P (A_{2}) + i \geq 3 \sum P (\emptyset) = P (A_{1}) + P (A_{2}) .

Step 2 — Decompose the union. For arbitrary

A, B

, write the three disjoint pieces:

A ∖ B, A \cap B, B ∖ A .

These satisfy

A = (A ∖ B) \cup (A \cap B)

B = (B ∖ A) \cup (A \cap B)

, and

A \cup B = (A ∖ B) \cup (A \cap B) \cup (B ∖ A)

, all unions disjoint. Step 3 — Apply finite additivity.

P (A) = P (A ∖ B) + P (A \cap B), P (B) = P (B ∖ A) + P (A \cap B) .

P (A \cup B) = P (A ∖ B) + P (A \cap B) + P (B ∖ A) .

Step 4 — Eliminate the slice probabilities. Substitute

P (A ∖ B) = P (A) - P (A \cap B)

and

P (B ∖ A) = P (B) - P (A \cap B)

P (A \cup B) = [P (A) - P (A \cap B)] + P (A \cap B) + [P (B) - P (A \cap B)] = P (A) + P (B) - P (A \cap B) . ■

Continuity of Probability Measures

Let $A_{1} \subseteq A_{2} \subseteq \dots$ be an increasing sequence of events with limit $A = ⋃_{n} A_{n}$ . Then

P (A) = n \to \infty lim P (A_{n}) = P (n = 1 ⋃ \infty A_{n})

⟳disjointify into rings

B_{n} = A_{n} ∖ A_{n - 1}

(with

A_{0} = \emptyset

), apply [6], and recognize the partial sums as

P (A_{n})

Dually, for a decreasing sequence $B_{1} \supseteq B_{2} \supseteq \dots$ with $B = ⋂_{n} B_{n}$ :

P (B) = n \to \infty lim P (B_{n})

⟳take complements to reduce to the increasing case [13], then apply [8].

These continuity properties are equivalent to countable additivity (given finite additivity), so they may be taken as an alternative form of Axiom III.

Andrey Nikolaevich Kolmogorov | 1903–1987

The Soviet mathematician Kolmogorov gave probability its modern, measure-theoretic foundation in his 1933 monograph Grundbegriffe der Wahrscheinlichkeitsrechnung ("Foundations of the Theory of Probability"). Before him, probability was a collection of techniques without a unifying axiomatic spine; rival foundations (frequency-based, à la von Mises) were heuristically motivated but logically fragile. Kolmogorov's stroke of genius was to recognize that Lebesgue's measure theory — developed for integration — was exactly the right language: a probability is simply a measure with total mass $1$ . This solved Hilbert's sixth problem (axiomatizing physics, including probability) for the probabilistic part, and made the law of large numbers, the central limit theorem, and stochastic processes rigorous objects. Kolmogorov also made foundational contributions to turbulence, complexity (Kolmogorov complexity), and dynamical systems.

Building Intuition

Think of $Ω$ as a lump of clay of total mass $1$ . An event is a region you carve out, and $P (A)$ is the mass of that region. The axioms then read naturally:

Non-negativity: regions can't have negative mass.
Normalization: the whole lump weighs exactly $1$ .
Countable additivity: if you slice the clay into non-overlapping pieces, their masses add up — and this works even for infinitely many slices.

The complement rule $P (A^{c}) = 1 - P (A)$ is "mass outside the region equals total minus mass inside." Inclusion–exclusion corrects for the fact that, if two regions overlap, naively adding their masses double-counts the overlap $A \cap B$ , so you subtract it once. The union bound [12] says: smearing pieces around so they overlap can only increase apparent total mass, so the mass of the union never exceeds the sum.

The reason we need a $σ$ -algebra rather than "all subsets" is that for a continuous lump (e.g. uniform clay on $[0, 1]$ ) some pathological "regions" cannot be assigned a consistent mass at all — these are the non-measurable sets, and we simply refuse to call them events.

The probability space (Ω, 𝓕, P): outcomes, events, and overlapping mass

Worked Examples

Two Dice — Computing Event Probabilities

Given. Roll two fair, distinguishable six-sided dice. The sample space is

Ω = {(i, j) : i, j \in {1, \dots, 6}}, ∣Ω∣ = 36,

with each outcome equally likely:

P ({ω}) = 1/36

. Find.

P (A)

where

A = {

sum

= 7}

, and

P (A \cup B)

where

B = {

first die

= 1}

. Step 1 — Event $A$ . Outcomes summing to

7

(1, 6), (2, 5), (3, 4), (4, 3), (5, 2), (6, 1)

— that's

6

outcomes.

P (A) = \frac{6}{36} = \frac{1}{6} .

Step 2 — Event $B$ . First die

= 1

(1, 1), \dots, (1, 6)

—

6

outcomes.

P (B) = \frac{6}{36} = \frac{1}{6} .

Step 3 — Overlap $A \cap B$ . First die

1

and sum

7

: only

(1, 6)

. So

P (A \cap B) = 1/36

. Step 4 — Inclusion–exclusion [11].

P (A \cup B) = \frac{6}{36} + \frac{6}{36} - \frac{1}{36} = \frac{11}{36} .

Answer.

P (A) = \frac{1}{6} \approx 0.167, P (A \cup B) = \frac{11}{36} \approx 0.306.

Continuous Sample Space — A Uniform Dart

Given. A dart lands uniformly at random in the unit square

Ω = [0, 1] \times [0, 1]

. Here

F

is the Borel

σ

-algebra and

P (A) = area (A)

, which is a valid probability measure since the total area is

1

. Find. (a)

P ({

dart lands within distance

0.5

of the origin

})

; (b)

P ({

dart lands exactly on the diagonal

y = x})

. Step 1 — Event (a). This is the quarter-disk of radius

0.5

inside the square:

P = area = \frac{1}{4} π r^{2} = \frac{1}{4} π (0.5)^{2} = \frac{π}{16} .

Step 2 — Numerically.

P \approx \frac{3.14159}{16} \approx 0.1963.

Step 3 — Event (b). The diagonal

y = x

is a line segment; as a subset of the plane it has area zero. Hence

P ({y = x}) = 0.

Key lesson. In a continuous space, probability zero does not mean impossible: the dart must land somewhere, every single point has probability zero, yet some point occurs. This is only consistent because uncountable unions are not covered by Axiom III. Answer.

(a) \frac{π}{16} \approx 0.196, (b) P = 0 (but not impossible).

Common Misconceptions

(1) "Every subset is an event." False for uncountable

Ω

. The power set of

[0, 1]

contains non-measurable sets (Vitali sets) to which no value

P

can be consistently assigned while respecting translation-invariance and countable additivity. This is why

F

is required. (2) "Probability $0$ means impossible; probability $1$ means certain." False in continuous spaces. Events of probability

0

can occur ("almost never"), and events of probability

1

can fail ("almost surely"). The correct phrasing is almost surely / almost never. (3) "Additivity holds for any union." It holds only for disjoint events (Axiom III), and only for countably many of them. For overlapping events use inclusion–exclusion [11]; for uncountable unions there is no general additivity rule. (4) " $P (A \cup B) = P (A) + P (B)$ always." Only when

A \cap B = \emptyset

. Forgetting the overlap term is the single most common elementary error. (5) "The sample space is unique." The same experiment can be modeled with different sample spaces depending on what is observed; only the induced probabilities of observable events must agree.

Finite Additivity Is Strictly Weaker Than Countable Additivity

Consider $Ω = N = {1, 2, 3, \dots}$ and suppose we demanded a "uniform" probability where every singleton has equal probability $p$ . If $p > 0$ , then countable additivity gives $P (Ω) = \sum_{n} p = \infty \neq = 1$ — contradiction. If $p = 0$ , then $P (Ω) = \sum_{n} 0 = 0 \neq = 1$ — contradiction. Conclusion: no uniform probability measure exists on a countably infinite set under Axiom III.

It is possible to build a finitely-additive measure on $N$ that assigns "density $0$ " to every finite set and is finitely additive — but it violates countable additivity. This shows that Axiom III (countable, not merely finite) is a genuine, non-trivial strengthening, and is exactly the ingredient that delivers continuity [13]–[14] and the limit theorems.

Connections

The axioms are the launching point for the entire subject. Conditioning on an event $B$ with $P (B) > 0$ defines Conditional Probability via $P (A ∣ B) = P (A \cap B) / P (B)$ , which inverts to give Bayes' Theorem and the Law of Total Probability. Mappings $X : Ω \to R$ that are $F$ -measurable are Random Variables, and the push-forward of $P$ along $X$ yields its Probability Distribution. Because $P$ is just a measure of total mass $1$ , all of Measure Theory and Lebesgue Integration applies, turning expectation into an integral $E [X] = \int_{Ω} X d P$ . The notion of Independence of events ( $P (A \cap B) = P (A) P (B)$ ) is a structural relation defined directly on the probability space, and indexed families of independent variables underlie the Law of Large Numbers and the Central Limit Theorem.

Real-World Applications

Statistical inference & hypothesis testing. $p$ -values, confidence intervals, and likelihoods are all probabilities of events in a sample space of possible data; the axioms guarantee internal consistency of these calculations.
Insurance and actuarial science. Pricing policies requires $P (claim)$ and $E [loss]$ ; inclusion–exclusion handles correlated perils (e.g., a flood damaging multiple insured houses).
Reliability engineering. A system's failure probability is computed from component-failure events using complements and the union bound [12] — e.g., $P (system works) = 1 - P (some critical component fails)$ .
Quantum mechanics (Born rule). The probability of measurement outcomes is $P (outcome) = ∣ ⟨ ψ ∣ ϕ ⟩ ∣^{2}$ ; these obey Kolmogorov's axioms on the sample space of measurement results for a fixed observable.
Cryptography & randomized algorithms. Security proofs and analyses of algorithms like randomized quicksort or Monte Carlo integration rest on bounding event probabilities (the union bound is ubiquitous).
Machine learning. Probabilistic models (Naive Bayes, Bayesian networks, generative models) are explicitly built on probability spaces; loss functions are expectations.
Epidemiology & risk assessment. Modeling disease transmission, false-positive/negative rates of medical tests, and pooled-testing strategies all reduce to manipulating event probabilities and conditionals.

Historical Development

1654 — Blaise Pascal and Pierre de Fermat correspond about gambling problems (the "problem of points"), founding the systematic study of chance.
1657 — Christiaan Huygens publishes De ratiociniis in ludo aleae, the first printed treatise on probability, introducing expectation.
1713 — Jacob Bernoulli's posthumous Ars Conjectandi states the (weak) law of large numbers, linking long-run frequency to probability.
1763 — Thomas Bayes' essay (published posthumously by Richard Price) introduces inverse/conditional reasoning, later formalized as Bayes' theorem.
1812 — Pierre-Simon Laplace's Théorie analytique des probabilités systematizes the classical (equally-likely-outcomes) definition and generating-function methods.
1900 — David Hilbert's sixth problem calls for the axiomatization of physics, explicitly including probability.
1909–1919 — Émile Borel and others develop the connection between measure theory and probability; Richard von Mises proposes a frequency-based (collective) foundation.
1933 — Andrey Kolmogorov publishes Grundbegriffe der Wahrscheinlichkeitsrechnung, giving the now-standard measure-theoretic axioms $(Ω, F, P)$ — the definition used in this article.
1938–1956 — Doob, Lévy, and Kolmogorov build modern stochastic-process theory (martingales, Brownian motion) atop the axioms, vindicating the framework's generality.

Subtleties & Limits

Foundational and philosophical frontiers.

The meaning of probability. Kolmogorov's axioms specify the mathematics but not the interpretation. The frequentist, Bayesian/subjectivist, and propensity interpretations all satisfy the same axioms yet disagree about what $P (A)$ means. There is no mathematical resolution; the choice affects statistical methodology.

Non-measurable sets and the Axiom of Choice. The existence of non-measurable subsets of $[0, 1]$ depends on the Axiom of Choice. Under the Axiom of Determinacy (incompatible with full Choice), every set of reals is Lebesgue measurable. Thus whether "all subsets are events" is even consistent depends on one's set-theoretic axioms.

Finitely-additive probability. Some researchers (de Finetti) argued countable additivity is too strong and that only finite additivity is normatively justified, since the strong law of large numbers can fail behavior in finitely-additive settings. This remains a live debate in the foundations of statistics.

Conditioning on null events. $P (A ∣ B)$ is undefined when $P (B) = 0$ , yet such conditioning is routine (e.g., conditioning a continuous variable on $X = x$ ). Resolving this rigorously requires regular conditional probabilities and the Borel–Kolmogorov paradox shows the answer can depend on how the null event is approached — a genuine subtlety with no fully canonical fix.

Quantum probability. The non-distributive lattice of quantum events does not embed into a single classical Kolmogorov space (Bell inequalities), motivating generalized (non-Kolmogorovian) probability theories.