Probability Models

A probability model is a precise mathematical description of a random experiment, consisting of a sample space of all possible outcomes together with a rule assigning probabilities to events. Key result: every valid model obeys the three Kolmogorov axioms — probabilities are non-negative, the whole sample space has probability 1, and the probability of mutually exclusive events adds. Used everywhere uncertainty appears: dice and cards, genetics, insurance, quality control, polling, and physics. Formalized by Andrey Kolmogorov in 1933, building on centuries of gambling mathematics.

A probability model packages a chance situation into three ingredients: the sample space $S$ (the set of every possible outcome), the collection of events (subsets of $S$ we can assign probability to), and a probability function $P$ that attaches a number between 0 and 1 to each event. The model matters because it turns vague statements like "this is likely" into computable quantities, allowing prediction, decision-making, and the entire edifice of statistics. A surprising fact is that the same abstract framework describes a coin flip, the decay of a radioactive atom, and the spread of a disease — the mathematics does not care what the outcomes physically mean. Once a model is fixed, every probability question has, in principle, a single correct numerical answer determined purely by the rules. The art lies in choosing the right model for a real situation.

The Core Insight

Randomness becomes mathematics the moment you commit to two things: what can happen (the sample space) and how the total probability of 1 is distributed across what can happen. Everything else — odds, expected values, conditional probabilities — is bookkeeping built on this foundation. A probability model is therefore not a guess about the future; it is a complete accounting system for uncertainty in which all the probabilities are forced to balance to 1.

Formal Statement

Sample Space and Events

The sample space $S$ is the set of all possible outcomes of a random experiment. An individual outcome is a sample point. An event $A$ is any subset of $S$ ; it "occurs" if the actual outcome belongs to $A$ .

$S$ — the entire sample space (the certain event)
$\emptyset$ — the empty set (the impossible event)
$A, B \subseteq S$ — events
$A \cup B$ — " $A$ or $B$ " (union)
$A \cap B$ — " $A$ and $B$ " (intersection)
$A^{c} = S ∖ A$ — "not $A$ " (complement)

Two events are mutually exclusive (disjoint) if $A \cap B = \emptyset$ — they cannot both occur.

Probability Function and the Kolmogorov Axioms

A probability function $P$ assigns to each event $A$ a real number $P (A)$ satisfying three axioms:

P (A) \geq 0 for every event A

⟳This is an axiom (a chosen rule); it encodes that probability is a measure of "amount of certainty," which cannot be negative.

P (S) = 1

⟳An axiom expressing that some outcome in

S

is certain to occur; it normalizes the total probability.

P (A_{1} \cup A_{2} \cup \dots) = i \sum P (A_{i}) when the A_{i} are pairwise mutually exclusive

⟳The countable additivity axiom; disjoint events contribute independently to the total, so their probabilities simply sum.

Consequences of the Axioms

From the three axioms alone, the following must hold for all events $A, B$ :

P (A^{c}) = 1 - P (A)

⟳

A

and

A^{c}

are disjoint with union

S

; apply [3] then [2].

P (\emptyset) = 0

⟳Set

A = S

in [4], since

S^{c} = \emptyset

0 \leq P (A) \leq 1

⟳Lower bound from [1]; upper bound because

P (A) = 1 - P (A^{c}) \leq 1

P (A^{c}) \geq 0

P (A \cup B) = P (A) + P (B) - P (A \cap B)

⟳The inclusion–exclusion principle; adding

P (A)

and

P (B)

double-counts the overlap

A \cap B

, so subtract it once.

If A \subseteq B then P (A) \leq P (B)

⟳Write

B = A \cup (B ∖ A)

as a disjoint union and apply [3] with [1].

Equally Likely Outcomes (Classical Model)

When $S$ is finite with $n$ outcomes that are all equally likely, the probability of any event $A$ containing $k$ favorable outcomes is

P (A) = \frac{∣ A ∣}{∣ S ∣} = \frac{number of favorable outcomes}{total number of outcomes} = \frac{k}{n}

⟳By symmetry each outcome has probability

1/ n

; by [3] the event's probability is the sum of

k

such terms.

Why It Holds / Derivation

Deriving the Complement and Inclusion–Exclusion Rules

We derive equations [4] and [7] from the axioms.

Step 1. For any event

A

, the events

A

and

A^{c}

are mutually exclusive (they share no outcome) and their union is all of

S

. Step 2. By additivity [3]:

P (A \cup A^{c}) = P (A) + P (A^{c})

. Step 3. Since

A \cup A^{c} = S

and

P (S) = 1

by [2], we get

P (A) + P (A^{c}) = 1

, hence

P (A^{c}) = 1 - P (A)

. This is [4]. Step 4. Now for [7], decompose

A \cup B

into three disjoint pieces: "only

A

" =

A \cap B^{c}

, "only

B

" =

A^{c} \cap B

, and "both" =

A \cap B

. Step 5. By additivity,

P (A \cup B) = P (A \cap B^{c}) + P (A^{c} \cap B) + P (A \cap B)

. Step 6. Also

P (A) = P (A \cap B^{c}) + P (A \cap B)

and

P (B) = P (A^{c} \cap B) + P (A \cap B)

, again by decomposing each into disjoint parts. Step 7. Add the two equations in Step 6:

P (A) + P (B) = P (A \cap B^{c}) + P (A^{c} \cap B) + 2 P (A \cap B)

. Step 8. Subtract

P (A \cap B)

from both sides and compare with Step 5:

P (A) + P (B) - P (A \cap B) = P (A \cup B)

. This is [7].

■

Andrey Kolmogorov | 1903–1987

The Soviet mathematician Andrey Nikolaevich Kolmogorov gave probability its modern rigorous foundation in his 1933 monograph Grundbegriffe der Wahrscheinlichkeitsrechnung ("Foundations of the Theory of Probability"). Before him, probability rested on intuitive notions of "equally likely cases" and frequency, which broke down for continuous and infinite situations. Kolmogorov's insight was to identify probability with measure theory: a sample space, a collection of measurable events (a σ-algebra), and a measure of total mass 1. This abstraction unified discrete and continuous probability and let mathematicians prove deep results rigorously. He also made fundamental contributions to turbulence, dynamical systems, and information theory.

Building Intuition

Picture a probability model as one liter of paint to be spread over a map of all possible outcomes. The map is the sample space; you must use exactly one liter (axiom [2]); you cannot apply negative paint (axiom [1]); and the paint over a region equals the sum of paint over its non-overlapping subregions (axiom [3]). An "event" is just an area on the map, and its probability is how much paint covers it. The complement rule [4] says the paint off a region equals one liter minus the paint on it. Inclusion–exclusion [7] is the obvious statement that when two regions overlap, adding their paint double-counts the overlap. Choosing a model means deciding how thickly to spread the paint — uniformly (classical model), or in some uneven pattern that better matches reality.

A probability model: total mass 1 distributed over the sample space, with two overlapping events

Worked Examples

Drawing a Card

Given: A standard 52-card deck is well shuffled; one card is drawn. Let

A

= "the card is a heart" and

B

= "the card is a face card (J, Q, K)". Find

P (A \cup B)

. Step 1 — Model. Sample space

S

= the 52 cards, all equally likely, so use [9] with

n = 52

. Step 2 — Count favorables. Hearts:

∣ A ∣ = 13

, so

P (A) = 13/52

. Face cards:

∣ B ∣ = 12

(3 per suit

\times

4 suits), so

P (B) = 12/52

. Step 3 — Overlap. Hearts that are also face cards: J♥, Q♥, K♥, so

∣ A \cap B ∣ = 3

and

P (A \cap B) = 3/52

. Step 4 — Inclusion–exclusion [7].

P (A \cup B) = \frac{13}{52} + \frac{12}{52} - \frac{3}{52} = \frac{22}{52} = \frac{11}{26} .

P (A \cup B) = \frac{11}{26} \approx 0.423

A Biased (Non-Uniform) Model

Given: A spinner has 3 outcomes — Red, Green, Blue — with

P (Red) = 0.5

P (Green) = 0.3

. Outcomes are mutually exclusive and exhaustive. Find

P (Blue)

and

P (not Red)

. Step 1 — Use normalization [2]. Probabilities must sum to 1:

P (Red) + P (Green) + P (Blue) = 1.

Step 2 — Solve for Blue.

P (Blue) = 1 - 0.5 - 0.3 = 0.2.

Step 3 — Complement rule [4].

P (not Red) = 1 - P (Red) = 1 - 0.5 = 0.5.

This is not a classical equally-likely model — the outcomes have unequal weights, yet the axioms still hold and the total is exactly 1.

P (Blue) = 0.2, P (not Red) = 0.5

Common Misconceptions

Probabilities are not always $1/ n$ . Equation [9] applies only when outcomes are equally likely. A common error is to list the sample space and assume uniform probabilities without justification. For the sum of two dice, the outcomes

{2, 3, \dots, 12}

are not equally likely — a 7 is six times as likely as a 2 — so the correct model uses the 36 equally likely ordered pairs, not the 11 sums. Always check whether your chosen outcomes deserve equal weight. The "gambler's fallacy." A probability model for independent trials carries no memory. After five heads in a row, a fair coin still has

P (heads) = 1/2

on the next flip. The coin does not "owe" you tails. Past outcomes do not change the model's probabilities for future independent trials.

Additivity Fails Without Disjointness

Suppose $P (rain Saturday) = 0.6$ and $P (rain Sunday) = 0.6$ . It is tempting to write $P (rain at least one day) = 0.6 + 0.6 = 1.2$ — but probabilities can never exceed 1 (by [6]). The error: axiom [3] requires mutually exclusive events, and "rain Saturday" and "rain Sunday" can both happen. The correct tool is [7], which subtracts the overlap $P (rain both days)$ , keeping the answer $\leq 1$ .

Connections

A probability model is the launching point for nearly all of probability and statistics. Once the model assigns probabilities, Conditional Probability asks how probabilities update given partial information, and Independence of Events characterizes when one event carries no information about another. Attaching numerical values to outcomes produces a Random Variable, whose long-run average is the Expected Value and whose spread is the Variance. Specific models — like the Binomial Distribution for repeated independent trials or the Normal Distribution for continuous measurements — are particular probability models with named structure. The classical model rests on Combinatorics for counting favorable outcomes.

Real-World Applications

Probability models underpin decision-making across science and society:

Insurance and actuarial science — Lloyd's of London and modern insurers price premiums using models of accident, mortality, and catastrophe probabilities.
Genetics — Mendelian inheritance is a probability model: a Punnett square assigns equally likely allele combinations, predicting trait ratios like 3:1.
Quality control — Manufacturers (e.g., Six Sigma programs at Motorola and GE) model defect rates to decide acceptance sampling and tolerances.
Epidemiology — Models of infection probability drove COVID-19 forecasting and vaccine trial design.
Polling and elections — Organizations like Gallup and FiveThirtyEight model voter behavior to attach probabilities to outcomes from sample data.
Casino and lottery design — Games are engineered around exact probability models to guarantee a house edge.
Physics — Radioactive decay and quantum measurement are modeled as genuinely random processes governed by probability distributions.

Historical Development

1654 — Blaise Pascal and Pierre de Fermat exchange letters on dividing stakes in an interrupted gambling game (the "problem of points"), founding mathematical probability.
1657 — Christiaan Huygens publishes De Ratiociniis in Ludo Aleae, the first printed treatise on probability, introducing expectation.
1713 — Jacob Bernoulli's posthumous Ars Conjectandi states the Law of Large Numbers, linking probability to long-run frequency.
1718 — Abraham de Moivre's The Doctrine of Chances develops approximations leading toward the normal curve.
1763 — Thomas Bayes's theorem on conditional probability is published posthumously, founding Bayesian inference.
1812 — Pierre-Simon Laplace's Théorie analytique des probabilités systematizes the classical "equally likely" definition.
1900 — David Hilbert lists the axiomatization of probability as part of his sixth problem.
1933 — Andrey Kolmogorov publishes the axiomatic, measure-theoretic foundation that defines the modern probability model.
1950s onward — Probability becomes central to information theory (Shannon), statistical mechanics, and computer simulation (Monte Carlo methods at Los Alamos).

Subtleties & Limits

The axioms guarantee a consistent model but never tell you which model is correct for a real phenomenon — this is the deep problem of interpreting probability. The frequentist view defines $P (A)$ as the limiting relative frequency over infinitely many repetitions, but no experiment is truly repeatable infinitely, and some events ("there is life on Europa") are not repeatable at all. The Bayesian view treats probability as a degree of belief, raising the question of where prior probabilities come from. On infinite or continuous sample spaces, not every subset can be assigned a probability — pathological "non-measurable" sets exist (related to the Banach–Tarski paradox), which is precisely why Kolmogorov restricted events to a σ-algebra. Choosing and justifying a model thus remains as much a matter of scientific and philosophical judgment as of mathematics.