Random Variables and Distributions

A random variable is a measurable function assigning numbers to outcomes of a random experiment; its distribution describes how probability is spread over those numbers. Key result: every random variable is fully characterized by its cumulative distribution function (CDF), and integrable functions of it have well-defined expectations. Applies wherever uncertainty is quantified — statistics, physics, finance, machine learning. Foundational since Kolmogorov's 1933 axiomatization.

A random variable $X$ is a function $X : Ω \to R$ from a sample space $Ω$ (equipped with a probability measure) to the real numbers, subject to a measurability condition that makes probabilities of events like ${X \leq x}$ well-defined. Its distribution is the induced probability measure on $R$ , encoded compactly by the cumulative distribution function $F_{X} (x) = P (X \leq x)$ . This concept matters because it translates messy, abstract outcomes (coin flips, particle positions, stock prices) into numbers we can average, compare, and compute with. The surprising fact: a single function $F_{X}$ — monotone, right-continuous, with limits 0 and 1 — captures everything probabilistic about $X$ , and conversely every such function arises from some random variable.

The Core Insight

A random variable separates the mechanism of randomness (the sample space $Ω$ and its measure) from the numerical observable we care about. Once we have the distribution of $X$ , the original $Ω$ can be forgotten entirely. All probabilistic questions about $X$ — its mean, variance, the chance it exceeds a threshold — depend only on the distribution, not on the underlying experiment. This is why a "fair die" and a "spinner landing in six equal sectors" are interchangeable: same distribution, same probabilistic content.

Formal Statement

Probability Space

A probability space is a triple $(Ω, F, P)$ where:

$Ω$ is the sample space (set of all outcomes),
$F$ is a $σ$ -algebra of subsets of $Ω$ (the events), closed under complement and countable union,
$P : F \to [0, 1]$ is a probability measure with $P (Ω) = 1$ and countable additivity.

P (i = 1 ⋃ \infty A_{i}) = i = 1 \sum \infty P (A_{i}), for disjoint A_{i} \in F .

⟳This is the countable additivity axiom itself; disjointness ensures no outcome is double-counted.

Random Variable

A random variable is a function $X : Ω \to R$ that is measurable: for every $x \in R$ ,

{ω \in Ω : X (ω) \leq x} \in F .

⟳We require preimages of

(- \infty, x]

to be events so that

P (X \leq x)

is defined; these sets generate the Borel

σ

-algebra, so all Borel preimages become measurable.

Here $X (ω)$ is the numerical value observed when outcome $ω$ occurs. The measurability condition guarantees every "interesting" set ${X \in B}$ for Borel $B$ is an event.

Cumulative Distribution Function

The CDF of $X$ is

F_{X} (x) = P (X \leq x) = P ({ω : X (ω) \leq x}) .

⟳Compose the measure

P

with the preimage from [2]; well-definedness is exactly the measurability requirement.

Every CDF satisfies: (i) non-decreasing, (ii) right-continuous, (iii) $lim_{x \to - \infty} F_{X} (x) = 0$ and $lim_{x \to + \infty} F_{X} (x) = 1$ .

Characterization of Distributions

A function $F : R \to [0, 1]$ is the CDF of some random variable if and only if it is non-decreasing, right-continuous, and has limits $0$ at $- \infty$ and $1$ at $+ \infty$ . The distribution (the induced measure $P_{X} (B) = P (X \in B)$ on Borel sets) is uniquely determined by $F$ .

Discrete and Continuous Types

Discrete: $X$ takes values in a countable set; described by a probability mass function (PMF) $p_{X} (x) = P (X = x)$ , with $\sum_{x} p_{X} (x) = 1$ .
Continuous (absolutely continuous): $F_{X}$ has a probability density function (PDF) $f_{X} \geq 0$ with

F_{X} (x) = \int_{- \infty}^{x} f_{X} (t) d t, \int_{- \infty}^{\infty} f_{X} (t) d t = 1.

⟳

F_{X}

absolutely continuous

\Rightarrow

by Lebesgue's differentiation theorem

f_{X} = F_{X}^{'}

exists a.e. and integrates back to

F_{X}

For continuous $X$ , $P (X = x) = 0$ for each point; probability lives in intervals: $P (a < X \leq b) = F_{X} (b) - F_{X} (a)$ .

Expectation and Variance

The expectation (mean) of $X$ , when it exists, is

E [X] = \int_{Ω} X d P = \int_{- \infty}^{\infty} x d F_{X} (x) = {\sum_{x} x p_{X} (x) \int_{- \infty}^{\infty} x f_{X} (x) d x discrete continuous

⟳The middle form is a Lebesgue–Stieltjes integral against

F_{X}

; it specializes to a sum or ordinary integral depending on whether

F_{X}

jumps or is smooth.

The variance measures spread:

Var (X) = E [(X - E [X])^{2}] = E [X^{2}] - (E [X])^{2} .

⟳Expand

(X - μ)^{2} = X^{2} - 2 μ X + μ^{2}

and use linearity with

E [X] = μ

constant.

Why It Holds / Derivation

Law of the Unconscious Statistician (LOTUS)

Claim: For a measurable

g : R \to R

E [g (X)] = \int_{- \infty}^{\infty} g (x) d F_{X} (x)

— i.e. we can integrate against the distribution of

X

without knowing

Ω

By definition $E [g (X)] = \int_{Ω} g (X (ω)) d P (ω)$ , an integral over the abstract space.
The random variable $X$ pushes $P$ forward to a measure $P_{X}$ on $R$ via $P_{X} (B) = P (X^{- 1} (B))$ .
Change of variables for pushforward measures: for any measurable $h \geq 0$ ,

\int_{Ω} h (X (ω)) d P (ω) = \int_{R} h (x) d P_{X} (x) .

This holds first for indicators

h = 1_{B}

(where it is exactly the definition of

P_{X}

), then for simple functions by linearity, then for general

h \geq 0

by monotone convergence, then for signed

h

by splitting into positive and negative parts.

Setting $h = g$ and recognizing $P_{X}$ as the measure with CDF $F_{X}$ gives

E [g (X)] = \int_{R} g (x) d P_{X} (x) = \int_{- \infty}^{\infty} g (x) d F_{X} (x) . ■

The phrase "unconscious statistician" reflects that practitioners use this formula instinctively, often unaware that a measure-theoretic theorem justifies it.

Andrey Kolmogorov | 1903–1987

Kolmogorov's 1933 monograph Grundbegriffe der Wahrscheinlichkeitsrechnung placed probability on measure-theoretic foundations, defining a random variable precisely as a measurable function on a probability space. Before him, probability lacked a rigorous axiomatic basis and was riddled with paradoxes (Bertrand's paradox, ambiguous "geometric probability"). Kolmogorov's framework unified discrete and continuous probability under one roof, made conditional expectation rigorous, and connected probability to analysis, enabling the modern theories of stochastic processes, ergodic theory, and information.

Building Intuition

Think of the sample space $Ω$ as a bag of all possible "states of the world," each with a probability weight. A random variable is a measuring instrument: you reach into the bag, pull out a state $ω$ , and read off a number $X (ω)$ . The distribution is the histogram you'd build by repeating this enormously many times.

The CDF $F_{X} (x)$ answers "what fraction of the weight sits at or below $x$ ?" — it sweeps from 0 (far left) to 1 (far right). A jump in $F_{X}$ means a lump of probability sits exactly at that point (discrete atom). A smooth rise means probability is smeared out (density). The steepness of the rise is exactly the density $f_{X}$ : where $F_{X}$ climbs fast, values are likely.

Expectation is the balance point of the histogram — where it would teeter on a knife-edge. Variance is the average squared distance from that balance point: small variance means a tall thin pile near the mean; large variance means a wide flat spread.

CDF and PDF of a continuous random variable: the density is the slope of the CDF

Worked Examples

Discrete: Expected value of a biased die

Given: A four-sided die with PMF

p (1) = 0.1, p (2) = 0.2, p (3) = 0.3, p (4) = 0.4

. Find:

E [X]

and

Var (X)

. Steps:

Check normalization: $0.1 + 0.2 + 0.3 + 0.4 = 1.0$ ✓
Mean via [5]:

E [X] = 1 (0.1) + 2 (0.2) + 3 (0.3) + 4 (0.4) = 0.1 + 0.4 + 0.9 + 1.6 = 3.0.

Second moment:

E [X^{2}] = 1 (0.1) + 4 (0.2) + 9 (0.3) + 16 (0.4) = 0.1 + 0.8 + 2.7 + 6.4 = 10.0.

Variance via [6]: $Var (X) = 10.0 - 3. 0^{2} = 10.0 - 9.0 = 1.0$ .

Answer:

E [X] = 3.0, Var (X) = 1.0, σ = 1.0

(dimensionless pip-count).

Continuous: Exponential waiting time

Given:

X \sim Exponential (λ)

with density

f (x) = λ e^{- λ x}

for

x \geq 0

λ = 2 hr^{- 1}

(e.g. customer arrivals). Find:

F_{X}

E [X]

, and

P (X > 1 hr)

. Steps:

CDF via [4]: $F_{X} (x) = \int_{0}^{x} 2 e^{- 2 t} d t = 1 - e^{- 2 x}$ for $x \geq 0$ .
Mean via [5] (integration by parts):

E [X] = \int_{0}^{\infty} x \cdot 2 e^{- 2 x} d x = \frac{1}{λ} = \frac{1}{2} = 0.5 hr .

Tail probability: $P (X > 1) = 1 - F_{X} (1) = e^{- 2 (1)} = e^{- 2} \approx 0.1353$ .

Answer:

E [X] = 0.5 hr, P (X > 1 hr) \approx 0.135

Common Misconceptions

A density is not a probability. For a continuous random variable,

f_{X} (x)

can exceed 1 (e.g. a Uniform

(0, 0.5)

density equals 2 everywhere on its support). Only integrals of

f_{X}

over intervals are probabilities. Likewise,

P (X = x) = 0

for continuous

X

does not mean the value

x

is impossible — it means it has zero measure, a subtle distinction. Impossible events have probability 0, but probability-0 events need not be impossible. Expectation may not exist. Not every random variable has a finite (or even defined) mean. The integral in [5] requires

E [∣ X ∣] < \infty

. When this fails, statements like "the average of

X

" are meaningless, and the Law of Large Numbers does not apply in its usual form.

Cauchy distribution: no mean

Let $X$ have the standard Cauchy density $f (x) = \frac{1}{π ( 1 + x ^{2} )}$ . Then

\int_{- \infty}^{\infty} ∣ x ∣ \frac{1}{π ( 1 + x ^{2} )} d x = \frac{2}{π} \int_{0}^{\infty} \frac{x}{1 + x ^{2}} d x = \frac{1}{π} [ln (1 + x^{2})]_{0}^{\infty} = \infty.

E [∣ X ∣] = \infty

and

E [X]

is undefined. Sample averages of Cauchy data do not converge to any number — they keep fluctuating with the same Cauchy distribution. This dramatically refutes the intuition that "averaging always stabilizes."

Connections

Random variables are the atoms from which the entire probabilistic edifice is built. The CDF connects to Cumulative Distribution Functions and Lebesgue–Stieltjes integration. Sums of independent random variables lead to Convolution of Distributions and the Central Limit Theorem. Conditional structure between several random variables is formalized in Joint Distributions and Independence, while the long-run behavior of averages appears in the Law of Large Numbers. The transform machinery — Moment Generating Functions and characteristic functions — uses LOTUS [3]–[5] to encode distributions analytically.

Real-World Applications

Insurance & actuarial science: Claim sizes modeled as heavy-tailed random variables (lognormal, Pareto); premiums set from $E [X]$ plus variance loading.
Quality control (Six Sigma): Manufacturing measurements treated as Normal random variables; defect rates computed from tail probabilities $P (∣ X - μ ∣ > 3 σ)$ .
Finance (Value at Risk): Portfolio returns modeled as random variables; the $α$ -quantile $F^{- 1} (α)$ gives the VaR threshold for regulatory capital.
Queueing & telecommunications: Inter-arrival times modeled as Exponential random variables (Poisson processes) to size servers and bandwidth.
Statistical physics: A particle's energy is a random variable distributed by the Boltzmann distribution $f (E) \propto e^{- E / k T}$ ; macroscopic quantities are expectations.
Machine learning: Model outputs are probability distributions over labels; cross-entropy loss is $E$ of $- lo g$ predicted probability under the true label distribution.
Epidemiology: Incubation periods and reproduction numbers modeled as Gamma/Negative-Binomial random variables to forecast outbreak spread.

Historical Development

1654 — Pascal and Fermat exchange letters on the problem of points, computing expected values of gambling outcomes — the implicit birth of expectation.
1713 — Jacob Bernoulli's Ars Conjectandi proves the (weak) Law of Large Numbers for binomial trials, the first limit theorem for a random variable's average.
1733–1812 — De Moivre and Laplace develop the Normal approximation to the binomial, an early Central Limit Theorem.
1809 — Gauss introduces the Normal distribution in the theory of measurement errors.
1900s — Borel and Lebesgue develop measure theory and the Lebesgue integral, the analytic tools probability needed.
1909 — Borel's strong law for binary digits foreshadows almost-sure convergence.
1933 — Kolmogorov's Grundbegriffe axiomatizes probability: random variable = measurable function, distribution = pushforward measure.
1937 — Cramér and Lévy systematize characteristic functions and distribution theory.
1950s onward — Doob, Itô, and others extend the framework to stochastic processes, turning random variables into the building blocks of modern probability.

Subtleties & Limits

Several deep subtleties surround even this elementary topic. (1) Non-measurable functions: with the Axiom of Choice one can construct sets (Vitali sets) and functions on $[0, 1]$ that are not measurable, hence not random variables — these have no well-defined probability, exposing the role of set-theoretic foundations. (2) Singular continuous distributions: there exist CDFs (e.g. the Cantor function) that are continuous yet have no density and no atoms — they are neither discrete nor absolutely continuous, defying the textbook dichotomy. (3) Existence of moments: characterizing which transforms (moment problem) uniquely determine a distribution from its moments is subtle — the lognormal distribution is famously not determined by its moments. (4) Infinite-dimensional analogues: defining "random functions" or measures on function spaces (Gaussian processes, Brownian motion) requires substantial extra machinery (Kolmogorov extension theorem) and motivates ongoing research in stochastic analysis.