Probability Rules

Probability Rules are the algebraic laws governing how probabilities of events combine. One-line statement: Probabilities lie between 0 and 1, sum to 1 over all outcomes, and combine via the addition rule (for "or"), the multiplication rule (for "and"), and the complement rule (for "not"). Where it applies: any situation involving uncertainty — dice, cards, weather, genetics, medical testing, insurance. Who/when: formalized by Pierre de Fermat and Blaise Pascal (1654), axiomatized by Andrey Kolmogorov (1933).

Probability rules are the set of fundamental laws that tell us how to compute the chance of combined or related events from the chances of simpler ones. They rest on a few axioms — every probability is a number between 0 and 1, the probability of something certain is 1 — and from these flow the addition, multiplication, and complement rules. These rules matter because virtually every quantitative prediction about an uncertain world (will it rain, will a patient test positive, will a stock rise) reduces to combining elementary probabilities correctly. A surprising fact: two events can each be likely, yet their joint occurrence can be very unlikely, and conversely rare-seeming coincidences (two people in a room of 23 sharing a birthday) are actually more likely than not — a direct consequence of how the multiplication rule compounds.

The Core Insight

Probability is a measure — a way of assigning a "size" between 0 and 1 to sets of outcomes. Because outcomes are sets, the entire subject reduces to the arithmetic of overlapping and disjoint sets. "Or" corresponds to union and uses addition (with a correction for overlap); "and" corresponds to intersection and uses multiplication (adjusted for dependence); "not" corresponds to the complement and uses subtraction from 1. Master these three correspondences and you can solve almost any introductory problem.

Formal Statement

Sample Space, Events, and Probability

The sample space $S$ is the set of all possible outcomes of an experiment. An event $A$ is any subset of $S$ . A probability function $P$ assigns to each event a number satisfying Kolmogorov's axioms:

0 \leq P (A) \leq 1

⟳An event cannot happen less often than never (0) or more often than always (1); this is an axiom, not derived.

P (S) = 1

⟳Some outcome in the sample space must occur, so the certain event has probability 1 — an axiom.

P (A \cup B) = P (A) + P (B) if A \cap B = \emptyset

⟳For mutually exclusive (disjoint) events, the count of favourable outcomes simply adds; this is the additivity axiom.

Symbols:

S

= sample space;

A, B

= events (subsets of

S

);

P (A)

= probability of

A

;

A \cup B

= "

A

B

" (union);

A \cap B

= "

A

and

B

" (intersection);

\emptyset

= empty set;

A^{c}

= complement of

A

(everything not in

A

Equally Likely Outcomes

When $S$ has $n$ equally likely outcomes and event $A$ contains $k$ of them, the classical definition gives:

P (A) = \frac{k}{n} = \frac{number of favourable outcomes}{total number of outcomes}

⟳Each outcome carries equal probability

1/ n

by symmetry; adding

k

of these via [3] gives

k / n

The Four Core Rules

From the axioms, the following derived rules hold for all events $A, B$ .

Complement Rule:

P (A^{c}) = 1 - P (A)

⟳

A

and

A^{c}

are disjoint and together form

S

; apply [3] then [2].

General Addition Rule:

P (A \cup B) = P (A) + P (B) - P (A \cap B)

⟳Adding

P (A)

and

P (B)

double-counts the overlap

A \cap B

; subtract it once.

Conditional Probability:

P (A ∣ B) = \frac{P ( A \cap B )}{P ( B )}, P (B) > 0

⟳Restricting to the world where

B

occurred rescales probabilities by dividing by

P (B)

General Multiplication Rule:

P (A \cap B) = P (A) P (B ∣ A) = P (B) P (A ∣ B)

⟳Rearrange [7]; the chance of both is the chance of one times the chance of the other given the first.

For independent events, $P (B ∣ A) = P (B)$ , so [8] simplifies:

P (A \cap B) = P (A) P (B) (independence)

⟳Independence means knowing

A

tells nothing about

B

, so

P (B ∣ A) = P (B)

; substitute into [8].

Why It Holds / Derivation

Deriving the General Addition Rule from the Axioms

We prove [6] using only axioms [1]–[3].

Step 1. Partition

A \cup B

into three disjoint pieces: outcomes in

A

only, in

B

only, and in both. Formally, write

A = (A \cap B^{c}) \cup (A \cap B)

, a disjoint union. Step 2. By additivity [3]:

P (A) = P (A \cap B^{c}) + P (A \cap B) .

Step 3. Similarly decompose

A \cup B = (A \cap B^{c}) \cup (A \cap B) \cup (B \cap A^{c})

, three mutually disjoint pieces. By [3]:

P (A \cup B) = P (A \cap B^{c}) + P (A \cap B) + P (B \cap A^{c}) .

Step 4. From Step 2,

P (A \cap B^{c}) = P (A) - P (A \cap B)

. Likewise

P (B \cap A^{c}) = P (B) - P (A \cap B)

. Step 5. Substitute into Step 3:

P (A \cup B) = [P (A) - P (A \cap B)] + P (A \cap B) + [P (B) - P (A \cap B)] .

Step 6. Simplify to obtain

P (A \cup B) = P (A) + P (B) - P (A \cap B)

, which is [6].

■

Andrey Kolmogorov | 1903–1987

The Russian mathematician Andrey Kolmogorov transformed probability from a collection of clever gambling tricks into a rigorous branch of mathematics. In his 1933 monograph Grundbegriffe der Wahrscheinlichkeitsrechnung ("Foundations of the Theory of Probability"), he showed that probability is just a special kind of measure theory — the mathematics of assigning sizes to sets. His three axioms (non-negativity, normalization to 1, and countable additivity) are exactly equations [1]–[3] above, and every rule in this article is a logical consequence of them. This unification let probability borrow the full power of real analysis, and it underpins modern statistics, quantum mechanics, and finance.

Building Intuition

Think of the sample space as a dartboard of total area 1. Each event is a region on the board, and its probability is just the fraction of the board it covers. Then:

Complement [5]: the chance of not hitting a region is the rest of the board — $1$ minus that region.
Addition [6]: to find the area covered by region $A$ or region $B$ , add their areas but subtract the overlap you counted twice.
Multiplication [8]: "and" shrinks things — the chance of landing in $A$ and then in a sub-region $B$ is a fraction of a fraction, so you multiply.

The key mental switch: "or" tends to make probabilities bigger (addition); "and" tends to make them smaller (multiplication).

Venn diagram of two overlapping events showing the addition rule

Worked Examples

Drawing a Card: Addition Rule

Given: A standard 52-card deck. Draw one card. Let

A

= "card is a King",

B

= "card is a Heart". Find

P (King or Heart)

. Step 1. Count favourable outcomes. There are 4 Kings, so

P (A) = 4/52

. There are 13 Hearts, so

P (B) = 13/52

. Step 2. Find the overlap. The King of Hearts is both, so

P (A \cap B) = 1/52

. Step 3. Apply the addition rule [6]:

P (A \cup B) = \frac{4}{52} + \frac{13}{52} - \frac{1}{52} = \frac{16}{52} .

Step 4. Simplify:

16/52 = 4/13 \approx 0.308

P (King or Heart) = \frac{4}{13} \approx 0.308

Note: had we forgotten to subtract the overlap, we'd wrongly get $17/52$ .

Two Draws: Multiplication Rule With and Without Replacement

Given: A bag has 5 red and 3 blue marbles (8 total). Draw two marbles. Find

P (both red)

(a) with replacement, (b) without replacement. Part (a) — with replacement (independent events). First draw:

P (red_{1}) = 5/8

. After replacing, the second draw is identical:

P (red_{2}) = 5/8

. By independence [9]:

P (both red) = \frac{5}{8} \times \frac{5}{8} = \frac{25}{64} \approx 0.391.

P_{with} = \frac{25}{64} \approx 0.391

Part (b) — without replacement (dependent events). First draw:

P (red_{1}) = 5/8

. Now 4 red remain out of 7. So

P (red_{2} ∣ red_{1}) = 4/7

. By the multiplication rule [8]:

P (both red) = \frac{5}{8} \times \frac{4}{7} = \frac{20}{56} = \frac{5}{14} \approx 0.357.

P_{without} = \frac{5}{14} \approx 0.357

The "without replacement" probability is lower because removing a red marble reduces the chance of a second red.

Common Misconceptions

Independent ≠ Mutually Exclusive

These two terms are routinely confused but are nearly opposite ideas.

Mutually exclusive events cannot both happen: $P (A \cap B) = 0$ . Use the simple addition rule.
Independent events do not affect each other: $P (A \cap B) = P (A) P (B)$ .

If two events with nonzero probability are mutually exclusive, they are necessarily dependent — because if $A$ happens, $B$ definitely cannot, so $A$ gives strong information about $B$ . Never assume an event combination is independent without checking; physical setup determines it.

The Gambler's Fallacy

Claim (false): "A fair coin has landed heads five times in a row, so tails is now 'due' and more likely on the next flip." Reality: Successive coin flips are independent [9]. The coin has no memory. The probability of tails on flip six is exactly

1/2

, regardless of history:

P (tails on flip 6 ∣ HHHHH) = P (tails) = \frac{1}{2} .

What is rare is getting six heads in a row from the start:

P = (1/2)^{6} = 1/64

. But once five heads have already occurred, that part is no longer uncertain — only the sixth flip remains, and it is a fresh

1/2

. Confusing the probability of a whole sequence with the probability of the next step is the gambler's fallacy.

Connections

Probability rules sit at the centre of a web of related topics. They generalize from Set Theory, since events are sets and the addition rule is exactly the inclusion–exclusion principle for set sizes. Conditional Probability (equation [7]) extends the multiplication rule and leads directly to Bayes' Theorem, the engine of modern statistical inference and machine learning. Counting favourable outcomes for equation [4] requires Combinatorics (permutations and combinations). When repeated independent trials are involved, the multiplication rule [9] gives rise to the Binomial Distribution, and averaging outcomes connects to Expected Value. Together these form the foundation of Descriptive Statistics and Inferential Statistics.

Real-World Applications

Medical diagnostics: Combining test sensitivity and disease prevalence via conditional probability [7] and Bayes' theorem to interpret a positive HIV or mammogram result.
Insurance and actuarial science: Lloyd's of London and modern insurers price premiums using the addition and multiplication rules over many independent risks.
Weather forecasting: A "30% chance of rain" is a probability; forecasters combine event probabilities for compound predictions (rain and wind).
Genetics (Mendelian inheritance): Punnett squares apply equation [4] and the multiplication rule to predict offspring genotype probabilities — e.g., $1/4$ chance of two recessive alleles.
Quality control: Manufacturers compute the probability that a batch passes inspection using independence [9] across many components.
Casino and lottery design: The expected loss in roulette and the astronomically small odds of a Powerball jackpot ( $\approx 1$ in 292 million) are pure applications of equation [4].
Reliability engineering: NASA computes the failure probability of a spacecraft as $1 - \prod (1 - p_{i})$ over independent component failures, using the complement and multiplication rules.

Historical Development

1564 (publ. 1663): Gerolamo Cardano writes Liber de Ludo Aleae, the first systematic treatise on gambling odds, anticipating equation [4].
1654: Blaise Pascal and Pierre de Fermat exchange letters solving the "problem of points" (how to divide stakes in an interrupted game), founding mathematical probability.
1657: Christiaan Huygens publishes De Ratiociniis in Ludo Aleae, the first printed probability textbook, introducing expected value.
1713: Jacob Bernoulli's Ars Conjectandi proves the Law of Large Numbers and formalizes the multiplication rule for independent trials.
1763: Thomas Bayes' essay on inverse probability is published posthumously, giving the conditional-probability formula [7] its inferential power.
1812: Pierre-Simon Laplace's Théorie analytique des probabilités consolidates the classical theory and the equally-likely definition [4].
1900: David Hilbert's sixth problem calls for the axiomatization of probability and physics.
1933: Andrey Kolmogorov publishes the measure-theoretic axioms [1]–[3], placing all probability rules on a rigorous footing.

Subtleties & Limits

The classical rules assume a well-defined sample space with assignable probabilities — but specifying these is not always straightforward. Three live issues:

What does probability mean? The frequentist view (probability = long-run relative frequency) and the Bayesian view (probability = degree of belief) give the same algebra [1]–[9] but disagree philosophically about single, non-repeatable events ("the probability it will rain tomorrow"). This interpretation debate remains unresolved.
Infinite and continuous sample spaces: When outcomes are uncountable (e.g., a randomly chosen real number in $[0, 1]$ ), individual outcomes have probability zero, and equation [4] fails. One needs measure theory and probability density functions — beyond the equally-likely counting model.
Ill-posed problems: The Bertrand paradox shows that "choose a random chord of a circle" yields different probabilities depending on the (unstated) method of randomization, revealing that "at random" must always be precisely defined.