Sample Space & Basic Probability: Building the Foundation

A sample space is the set of all possible outcomes of a random experiment; probability assigns each outcome a number between 0 and 1 measuring how likely it is. For equally likely outcomes, the probability of an event equals (favourable outcomes) ÷ (total outcomes). This framework underlies all of statistics, gambling, genetics, and risk analysis, and was formalised by Pascal, Fermat, and Laplace between the 1650s and 1810s.

A random experiment is any process whose outcome cannot be predicted with certainty in advance but whose set of possible outcomes is known. The collection of every possible outcome is the sample space, usually written $S$ or $Ω$ , and any subset of it is called an event. A probability measure is a rule that assigns to each event a number expressing its likelihood, obeying three simple axioms. The surprising fact is that this entire rich subject — capable of describing quantum mechanics, insurance markets, and machine learning — rests on just three short rules first written down clearly by Kolmogorov only in 1933, long after people had been computing probabilities successfully for nearly three centuries.

The Core Insight

Probability is built in two layers. First, list every possible outcome — the sample space. Second, attach weights (probabilities) to outcomes so the weights are non-negative and sum to 1. Once these two steps are done, the probability of any event is simply the sum of the weights of the outcomes inside it. Everything else — addition rules, complements, conditional probability — is bookkeeping on this foundation.

Formal Statement

Random Experiment, Sample Space, Event

A random experiment is a repeatable procedure with a well-defined set of possible results.

The sample space $S$ (or $Ω$ ) is the set of all distinct possible outcomes. Each individual outcome $ω \in S$ is called a sample point or elementary event.

An event $A$ is any subset of the sample space, $A \subseteq S$ . We say "event $A$ occurs" if the actual outcome $ω$ satisfies $ω \in A$ .

The certain event is $S$ itself.
The impossible event is the empty set $\emptyset$ .
An elementary event is a single-outcome subset ${ω}$ .

Probability Measure — Kolmogorov's Axioms

A probability measure $P$ assigns to each event $A \subseteq S$ a real number $P (A)$ satisfying:

P (A) \geq 0 for every event A

⟳This is an axiom (non-negativity); probabilities cannot be negative because they measure a proportion of likelihood.

P (S) = 1

⟳An axiom (normalisation); something in the sample space must happen, so the total probability is exactly 1.

P (i ⋃ A_{i}) = i \sum P (A_{i}) if A_{i} \cap A_{j} = \emptyset for i \neq = j

⟳The additivity axiom; for mutually exclusive (disjoint) events, the probability of "at least one occurring" is the sum of their separate probabilities.

Here $A_{i} \cap A_{j} = \emptyset$ means the events are mutually exclusive — they cannot both occur.

Classical (Laplace) Probability for Equally Likely Outcomes

If $S$ is finite with $n = ∣ S ∣$ outcomes that are all equally likely, then for any event $A$ containing $∣ A ∣$ outcomes:

P (A) = \frac{∣ A ∣}{∣ S ∣} = \frac{number of favourable outcomes}{total number of outcomes}

⟳Assign each of

n

outcomes equal weight

1/ n

(forced by [2] and [3]); summing the weights of the

∣ A ∣

outcomes in

A

gives

∣ A ∣/ n

Basic Consequences

From axioms [1]–[3], the following hold for all events $A, B \subseteq S$ :

P (A^{c}) = 1 - P (A)

⟳

A

and its complement

A^{c}

are disjoint and their union is

S

; apply [3] then [2].

P (\emptyset) = 0

⟳Take

A = S

in [5], since

S^{c} = \emptyset

and

P (S) = 1

0 \leq P (A) \leq 1

⟳Lower bound from [1]; upper bound because

A \subseteq S

forces

P (A) \leq P (S) = 1

via monotonicity.

P (A \cup B) = P (A) + P (B) - P (A \cap B)

⟳The inclusion–exclusion principle; outcomes in the overlap

A \cap B

get counted twice when adding

P (A) + P (B)

, so subtract them once.

Why It Holds / Derivation

:::proof[Derivation of the Addition Rule [8]] We prove $P (A \cup B) = P (A) + P (B) - P (A \cap B)$ from the axioms.

Step 1. Decompose

A \cup B

into three mutually exclusive pieces:

$A ∖ B$ (in $A$ only),
$A \cap B$ (in both),
$B ∖ A$ (in $B$ only).

These are pairwise disjoint and their union is exactly $A \cup B$ .

Step 2. By the additivity axiom [3]:

P (A \cup B) = P (A ∖ B) + P (A \cap B) + P (B ∖ A) .

Step 3. Note

A = (A ∖ B) \cup (A \cap B)

, disjointly, so

P (A) = P (A ∖ B) + P (A \cap B)

, giving

P (A ∖ B) = P (A) - P (A \cap B)

. Similarly

P (B ∖ A) = P (B) - P (A \cap B)

. Step 4. Substitute into Step 2:

P (A \cup B) = [P (A) - P (A \cap B)] + P (A \cap B) + [P (B) - P (A \cap B)] .

Step 5. Simplify:

P (A \cup B) = P (A) + P (B) - P (A \cap B) . ■

:::proof[Derivation of the Complement Rule [5]] Step 1. $A$ and $A^{c}$ are disjoint by definition, and $A \cup A^{c} = S$ .

Step 2. By additivity [3]:

P (A) + P (A^{c}) = P (S)

. Step 3. By normalisation [2]:

P (S) = 1

, so

P (A) + P (A^{c}) = 1

. Step 4. Rearrange:

P (A^{c}) = 1 - P (A)

■

Andrey Kolmogorov | 1903–1987

The Soviet mathematician Andrey Nikolaevich Kolmogorov gave probability its modern rigorous footing in his 1933 monograph Grundbegriffe der Wahrscheinlichkeitsrechnung ("Foundations of the Theory of Probability"). He recognised that probability is a special case of measure theory: a sample space with a "measure" that assigns total mass 1. Before him, probability was a collection of clever but informal techniques; after him, it became a fully axiomatic branch of pure mathematics. Kolmogorov also made deep contributions to turbulence, dynamical systems, and algorithmic complexity.

Building Intuition

Think of the sample space as a pie and probability as how the pie is sliced. The whole pie has area 1 (axiom [2]). Each outcome takes a slice whose size is its probability, and no slice can have negative area (axiom [1]). An event is a group of slices, and its probability is just the combined area (axiom [3]).

For a fair die, the pie is cut into 6 equal slices, each $\frac{1}{6}$ . The event "even number" $= {2, 4, 6}$ is three slices, area $\frac{3}{6} = \frac{1}{2}$ .

The crucial mental move is separating the listing from the weighting. First ask: what can happen? Then ask: how likely is each? Many errors come from skipping the first question — for instance, forgetting that rolling two dice has 36 outcomes, not 11.

The sample space of two dice as a 6×6 grid; the diagonal band marks outcomes summing to 7.

Worked Examples

Drawing a Card from a Standard Deck

Given. A well-shuffled standard deck of 52 cards. Draw one card at random. Find the probability the card is either a king or a heart. Step 1 — Sample space. All 52 cards are equally likely, so

∣ S ∣ = 52

. Step 2 — Define events.

$K$ = "card is a king", $∣ K ∣ = 4$ .
$H$ = "card is a heart", $∣ H ∣ = 13$ .
$K \cap H$ = "king of hearts", $∣ K \cap H ∣ = 1$ .

Step 3 — Individual probabilities using [4]:

P (K) = \frac{4}{52}, P (H) = \frac{13}{52}, P (K \cap H) = \frac{1}{52} .

Step 4 — Addition rule [8] (overlapping events, so subtract the intersection):

P (K \cup H) = \frac{4}{52} + \frac{13}{52} - \frac{1}{52} = \frac{16}{52} .

Step 5 — Simplify.

\frac{16}{52} = \frac{4}{13}

P (king or heart) = \frac{4}{13} \approx 0.3077

Sum of Two Fair Dice

Given. Roll two fair six-sided dice. Find (a)

P (sum = 7)

and (b)

P (sum \geq 10)

. Step 1 — Sample space. Ordered pairs

(d_{1}, d_{2})

, so

∣ S ∣ = 6 \times 6 = 36

equally likely outcomes. Step 2 (a) — Event sum = 7. Favourable pairs:

(1, 6), (2, 5), (3, 4), (4, 3), (5, 2), (6, 1)

→

∣ A ∣ = 6

P (sum = 7) = \frac{6}{36} = \frac{1}{6} \approx 0.1667.

Step 3 (b) — Event sum ≥ 10. Means sum = 10, 11, or 12 (mutually exclusive).

Sum 10: $(4, 6), (5, 5), (6, 4)$ → 3 outcomes
Sum 11: $(5, 6), (6, 5)$ → 2 outcomes
Sum 12: $(6, 6)$ → 1 outcome

Total favourable $= 3 + 2 + 1 = 6$ (by additivity [3]).

P (sum \geq 10) = \frac{6}{36} = \frac{1}{6} \approx 0.1667.

P (sum = 7) = \frac{1}{6}, P (sum \geq 10) = \frac{1}{6}

Common Misconceptions

Outcomes Are Not Always Equally Likely

Formula [4] only applies when every outcome in $S$ is equally likely. A common blunder is to list outcomes by result rather than by equally likely cases. For two dice, the sums ${2, 3, \dots, 12}$ form a valid sample space of 11 outcomes — but they are not equally likely, so $P (sum = 7) \neq = \frac{1}{11}$ . The correct equally likely sample space is the 36 ordered pairs. Always choose your sample space so the counting formula is justified, or weight the outcomes explicitly.

The Gambler's Fallacy

Independent random trials have no memory. After five consecutive heads, the probability of heads on the next fair toss is still $\frac{1}{2}$ , not lower. The coin does not "owe" you a tail. The mistaken belief that past outcomes influence future independent ones is the gambler's fallacy, and it has cost countless people their savings. See Independent Events.

Why "Equally Likely" Can't Be Assumed

Consider tossing two coins and recording the number of heads: the sample space is ${0, 1, 2}$ . A naïve student might say each has probability $\frac{1}{3}$ .

But list the equally likely raw outcomes: ${H H, H T, T H, T T}$ , four cases.

$0$ heads: only $T T$ → $\frac{1}{4}$
$1$ head: $H T, T H$ → $\frac{2}{4} = \frac{1}{2}$
$2$ heads: only $H H$ → $\frac{1}{4}$

So $P (1 head) = \frac{1}{2}$ , double the naïve guess. The lesson: a valid sample space need not have equally likely members — you must verify it.

Connections

Sample spaces and basic probability are the gateway to the whole subject. The addition rule [8] generalises to the Inclusion-Exclusion Principle for many events. Restricting the sample space to a known event leads directly to Conditional Probability and Bayes' Theorem. When outcomes carry numerical values, the weighting becomes a Random Variable and its Probability Distribution. Counting the size of events efficiently requires Combinatorics (permutations and combinations). Finally, the frequentist justification that observed relative frequencies approach true probabilities is made precise by the Law of Large Numbers.

Real-World Applications

Insurance & actuarial science. Premiums are computed from the probability of claims (death, accident, fire) across a sample space of policyholders.
Genetics — Punnett squares. The sample space of allele combinations from two parents predicts offspring trait probabilities (e.g. $\frac{3}{4}$ dominant for a monohybrid cross).
Medical screening. Test sensitivity and false-positive rates are probabilities over the sample space of {disease, no disease} × {positive, negative}.
Lotteries and casinos. Expected payouts and house edges are direct applications of [4] over enormous sample spaces (e.g. $(6 49)$ lottery combinations).
Quality control. A factory estimates the probability a randomly sampled item is defective to decide whether to accept a batch.
Weather forecasting. "30% chance of rain" is a probability assigned over an ensemble (sample space) of simulated atmospheric outcomes.
Cryptography & computing. Random number generators and hash collisions are analysed via the sample space of possible keys or outputs.

Historical Development

1564 (pub. 1663). Gerolamo Cardano writes Liber de Ludo Aleae, the first systematic treatment of dice odds, anticipating equally likely outcomes.
1654. Blaise Pascal and Pierre de Fermat exchange letters solving the problem of points (how to divide stakes in an interrupted game), founding mathematical probability.
1657. Christiaan Huygens publishes De Ratiociniis in Ludo Aleae, the first printed probability textbook, introducing expected value.
1713. Jacob Bernoulli's posthumous Ars Conjectandi proves the first Law of Large Numbers, linking probability to long-run frequency.
1733–1738. Abraham de Moivre derives the normal approximation to the binomial, foreshadowing the bell curve.
1812. Pierre-Simon Laplace's Théorie analytique des probabilités gives the classical definition [4] and treats probability as a tool for reasoning under uncertainty.
1900s. The frequentist (von Mises) and subjective/Bayesian (Ramsey, de Finetti) interpretations are debated.
1933. Andrey Kolmogorov publishes the axioms [1]–[3], unifying probability with measure theory and giving the modern foundation.

Subtleties & Limits

The neat picture of a finite, equally likely sample space breaks down in subtle cases. When $S$ is uncountably infinite (e.g. picking a random real number in $[0, 1]$ ), individual outcomes have probability zero, yet events still have positive probability — handled by Measure Theory and probability densities, not sums. Moreover, on such spaces one cannot consistently assign a probability to every subset (non-measurable sets exist, as shown by the Vitali and Banach–Tarski constructions), so we restrict to a $σ$ -algebra of "nice" events. A deeper, still-debated question is the interpretation of probability itself: is $P (A)$ an objective long-run frequency, a degree of rational belief (Bayesian), or a propensity of the physical system? This is a live issue in the foundations of statistics and quantum mechanics, where probabilities appear irreducibly fundamental.