Random Variables

A random variable is a measurable function that assigns a real number to each outcome of a random experiment. Key result in words: it transports probability from an abstract sample space onto the real line, where it is described by a distribution function, mass function, or density. Where it applies: all of probability theory, statistics, statistical physics, finance, information theory, and machine learning. Who/when: crystallized through Chebyshev and Kolmogorov (1933), building on Gauss, Laplace, and the measure theory of Borel and Lebesgue.

A random variable $X$ is not "random" and not "variable" in the everyday sense — it is a deterministic function $X : Ω \to R$ from a sample space $Ω$ to the real numbers, satisfying a measurability condition that guarantees probabilities like $P (X \leq x)$ are well-defined. Random variables matter because they let us replace clumsy reasoning about events in an arbitrary sample space with familiar calculus and algebra on the real line: we add them, take expectations, compute variances, and study their limiting behavior. The single most surprising fact is that the function itself contains all the structure, yet for almost all practical purposes we may forget $Ω$ entirely and work only with the distribution $X$ induces — two completely different experiments can yield identical random variables. This abstraction is what makes the Law of Large Numbers and Central Limit Theorem clean universal statements rather than ad hoc tricks.

The Core Insight

A random variable is the bridge between the messy, problem-specific world of outcomes and the clean, universal world of real numbers. Once you have $X$ , probability becomes the study of how mass is distributed along $R$ . Everything — expectation, variance, independence, convergence — is a statement about that distribution. The sample space becomes scaffolding you can discard.

Formal Statement

Probability Space

A probability space is a triple $(Ω, F, P)$ where:

$Ω$ is the sample space (set of all outcomes $ω$ ),
$F$ is a $σ$ -algebra of subsets of $Ω$ (the events), closed under complement and countable union,
$P : F \to [0, 1]$ is a probability measure with $P (Ω) = 1$ and countable additivity.

The axioms of countable additivity read:

P (i = 1 ⋃ \infty A_{i}) = i = 1 \sum \infty P (A_{i}), A_{i} \cap A_{j} = \emptyset (i \neq = j) .

⟳this is Kolmogorov's third axiom; it is an assumption, not a theorem, chosen to make limits of probabilities behave.

Random Variable

A random variable is a function $X : Ω \to R$ that is $F$ -measurable: for every $x \in R$ ,

X^{- 1} ((- \infty, x]) = {ω \in Ω : X (ω) \leq x} \in F .

⟳we require the preimage of every half-line to be an event so that

P (X \leq x)

makes sense; preimages of all Borel sets then follow because

{(- \infty, x]}

generates the Borel

σ

-algebra.

Symbols: $ω$ = an outcome; $X (ω)$ = its numerical value; $X^{- 1} (B) = {ω : X (ω) \in B}$ = preimage; $F$ = the events for which probability is defined.

Distribution and CDF

The cumulative distribution function (CDF) of $X$ is

F_{X} (x) = P (X \leq x) = P ({ω : X (ω) \leq x}) .

⟳apply

P

to the measurable set guaranteed by equation [2].

F_{X}

is non-decreasing, right-continuous, with

lim_{x \to - \infty} F_{X} (x) = 0

and

lim_{x \to + \infty} F_{X} (x) = 1

. The law / distribution of

X

is the pushforward measure

P_{X} (B) = P (X^{- 1} (B))

on the Borel sets.

Discrete vs Continuous

Discrete:

X

takes values in a countable set; described by a probability mass function (PMF)

p_{X} (x) = P (X = x)

with

\sum_{x} p_{X} (x) = 1

. (Absolutely) continuous: there exists a probability density function (PDF)

f_{X} \geq 0

with

F_{X} (x) = \int_{- \infty}^{x} f_{X} (t) d t, \int_{- \infty}^{\infty} f_{X} (t) d t = 1.

⟳this is the Radon–Nikodym derivative of

P_{X}

with respect to Lebesgue measure, when it exists;

f_{X} = F_{X}^{'}

wherever

F_{X}

is differentiable.

Expectation and Variance

For an integrable random variable, the expectation is

E [X] = \int_{Ω} X (ω) d P (ω) = \int_{- \infty}^{\infty} x d F_{X} (x),

⟳the abstract Lebesgue integral over

Ω

equals the Lebesgue–Stieltjes integral over

R

by the change-of-variables (pushforward) theorem.

reducing to $\sum_{x} x p_{X} (x)$ (discrete) or $\int x f_{X} (x) d x$ (continuous). The variance is

Var (X) = E [(X - E [X])^{2}] = E [X^{2}] - (E [X])^{2} .

⟳expand

(X - μ)^{2} = X^{2} - 2 μ X + μ^{2}

and use linearity of expectation with

μ = E [X]

Law of the Unconscious Statistician

For a measurable function $g : R \to R$ ,

E [g (X)] = \int_{- \infty}^{\infty} g (x) d F_{X} (x),

⟳

g (X)

is itself a random variable; apply the pushforward of

P

under

X

, so no need to know the distribution of

g (X)

explicitly.

Why It Holds / Derivation

Expectation as a Distribution Integral

We justify equation [5]: that the integral over the abstract space $Ω$ equals an integral over $R$ using only the law $P_{X}$ .

Indicators. For $X = 1_{A}$ with $A \in F$ , $\int_{Ω} 1_{A} d P = P (A)$ . As a real-valued variable this $X$ takes value $1$ on $A$ and $0$ elsewhere, and $\int x d F_{X} = 0 \cdot P (A^{c}) + 1 \cdot P (A) = P (A)$ . The identity holds.

Simple functions. Let $X = \sum_{k = 1}^{n} a_{k} 1_{A_{k}}$ with disjoint $A_{k}$ . By linearity of the integral, $\int_{Ω} X d P = \sum_{k} a_{k} P (A_{k})$ , which equals $\sum_{k} a_{k} P_{X} ({a_{k}}) = \int x d F_{X}$ .

Non-negative $X$ . Choose simple functions $X_{n} ↑ X$ pointwise. By the Monotone Convergence Theorem, $\int_{Ω} X_{n} d P \to \int_{Ω} X d P$ , and correspondingly on $R$ , giving the identity for non-negative $X$ .

General $X$ . Write $X = X^{+} - X^{-}$ with $X^{\pm} \geq 0$ . If at least one of $E [X^{+}], E [X^{-}]$ is finite, subtract to obtain equation [5]. $■$

Andrey Kolmogorov | 1903–1987

Kolmogorov's 1933 monograph Grundbegriffe der Wahrscheinlichkeitsrechnung placed probability on the rigorous footing of measure theory. His decisive move was to define a random variable as a measurable function on an abstract probability space, severing probability from any particular interpretation (frequency, betting, physical) and making it a branch of analysis. This let him prove the strong law of large numbers and the zero–one law with full rigor, and it remains the framework every undergraduate course silently assumes.

Building Intuition

Think of $Ω$ as a bag of tickets, one per outcome $ω$ , each printed with a number $X (ω)$ . Drawing a ticket at random (weighted by $P$ ) and reading its number is the random variable. The measurability condition just says: for any threshold $x$ , the collection of tickets reading " $\leq x$ " is one you're allowed to bet on (it lies in $F$ ).

Now imagine collapsing all tickets with the same number into one pile. The piles, with their total weights, are the distribution. Two utterly different bags (rolling a die vs. spinning a 6-sector wheel) can produce the same piles — hence the same random variable in distribution. This is why we routinely "forget $Ω$ ": all probabilistic questions about $X$ alone are answered by the pile weights, i.e., by $F_{X}$ .

CDF and PDF of a continuous random variable: the CDF is the running area under the density.

<svg viewBox="0 0 520 320" xmlns="http://www.w3.org/2000/svg">
<rect width="520" height="320" fill="#0a0e17"/>
<text x="260" y="22" fill="#e2e8f0" font-size="15" text-anchor="middle">Density f(x) and CDF F(x)</text>
<!-- axes -->
<line x1="50" y1="160" x2="480" y2="160" stroke="#94a3b8" stroke-width="1.5"/>
<line x1="50" y1="290" x2="50" y2="40" stroke="#94a3b8" stroke-width="1.5"/>
<text x="486" y="164" fill="#e2e8f0" font-size="12">x</text>
<!-- density curve (bell) on lower region -->
<path d="M50,288 C150,288 180,180 230,180 C280,180 300,288 480,288" fill="none" stroke="#d4a853" stroke-width="2.5"/>
<text x="250" y="172" fill="#d4a853" font-size="12" text-anchor="middle">f(x)</text>
<!-- shaded area up to x0 -->
<path d="M50,288 C150,288 180,180 230,180 L230,288 Z" fill="#d4a853" opacity="0.22"/>
<line x1="230" y1="288" x2="230" y2="160" stroke="#94a3b8" stroke-width="1" stroke-dasharray="4 3"/>
<text x="230" y="305" fill="#e2e8f0" font-size="12" text-anchor="middle">x₀</text>
<text x="135" y="250" fill="#e2e8f0" font-size="11">area = F(x₀)</text>
<!-- CDF curve upper -->
<path d="M50,150 C150,150 180,120 230,95 C280,70 330,52 480,48" fill="none" stroke="#94a3b8" stroke-width="2"/>
<text x="400" y="60" fill="#e2e8f0" font-size="12">F(x) → 1</text>
<circle cx="230" cy="95" r="3.5" fill="#d4a853"/>
<line x1="50" y1="95" x2="230" y2="95" stroke="#94a3b8" stroke-width="0.8" stroke-dasharray="3 3"/>
<text x="20" y="99" fill="#e2e8f0" font-size="11">F(x₀)</text>
<text x="55" y="55" fill="#e2e8f0" font-size="11">F(x): CDF (top)</text>
<text x="55" y="200" fill="#e2e8f0" font-size="11">f(x): density (bottom)</text>
</svg>

Worked Examples

A Fair Die — Discrete

Given: Roll a fair six-sided die.

Ω = {1, 2, 3, 4, 5, 6}

, each outcome probability

1/6

. Define

X (ω) = ω

(the face value). Steps:

PMF: $p_{X} (k) = 1/6$ for $k = 1, \dots, 6$ .
Expectation, eqn [5]: $E [X] = \sum_{k = 1}^{6} k \cdot \frac{1}{6} = \frac{1}{6} (1 + 2 + 3 + 4 + 5 + 6) = \frac{21}{6} = 3.5$ .
$E [X^{2}] = \frac{1}{6} (1 + 4 + 9 + 16 + 25 + 36) = \frac{91}{6} \approx 15.1667$ .
Variance, eqn [6]: $Var (X) = \frac{91}{6} - (3.5)^{2} = 15.1667 - 12.25 = 2.9167$ .

Answer:

E [X] = 3.5 (pips), Var (X) = \frac{35}{12} \approx 2.917 (pips^{2})

:::example[Uniform on [0,2] — Continuous] Given: $X$ uniform on $[0, 2]$ , so $f_{X} (x) = \frac{1}{2}$ for $0 \leq x \leq 2$ , else $0$ .

Steps:

Check normalization, eqn [4]: $\int_{0}^{2} \frac{1}{2} d x = 1$ . ✓
CDF, eqn [3]: $F_{X} (x) = \int_{0}^{x} \frac{1}{2} d t = x /2$ for $0 \leq x \leq 2$ .
Expectation, eqn [5]: $E [X] = \int_{0}^{2} x \cdot \frac{1}{2} d x = \frac{1}{2} \cdot \frac{x ^{2}}{2}_{0}^{2} = \frac{1}{2} \cdot 2 = 1$ .
$E [X^{2}] = \int_{0}^{2} x^{2} \cdot \frac{1}{2} d x = \frac{1}{2} \cdot \frac{x ^{3}}{3}_{0}^{2} = \frac{1}{2} \cdot \frac{8}{3} = \frac{4}{3}$ .
Variance, eqn [6]: $Var (X) = \frac{4}{3} - 1^{2} = \frac{1}{3}$ .

Answer:

E [X] = 1 (units), Var (X) = \frac{1}{3} \approx 0.333 (units^{2})

Common Misconceptions

"A random variable is a number" — false. It is a function

X : Ω \to R

. The number appears only after an outcome is drawn. Conflating the function with one of its values causes endless confusion when manipulating expressions like

E [X]

(a fixed number) versus

X

(a function). "Continuous means $P (X = x) > 0$ for some $x$ " — false. For a continuous random variable

P (X = a) = 0

for every single point

a

; probability lives in intervals, not points. Density

f_{X} (a)

is not a probability — it can exceed

1

. " $E [g (X)] = g (E [X])$ " — generally false (that is Jensen's inequality territory). Equality holds only for affine

g

A Function That Is Not a Random Variable

Let $Ω = [0, 1]$ but take a coarse $σ$ -algebra $F = {\emptyset, [0, \frac{1}{2}), [\frac{1}{2}, 1], Ω}$ . Define $X (ω) = ω$ . Then ${X \leq \frac{1}{4}} = [0, \frac{1}{4}]$ , which is not in $F$ . So $X$ fails the measurability condition [2] and is not a random variable with respect to this $F$ — even though it is a perfectly ordinary function. Measurability is relative to the chosen $σ$ -algebra, not an intrinsic property of the formula.

Connections

Random variables sit at the hub of probability. The notion of independence is defined via factorization of joint distributions of several random variables, leading to Joint Distributions. Sequences of them and their limiting behavior produce the Law of Large Numbers and Central Limit Theorem. The expectation operator is a special case of the Lebesgue Integration machinery, and the existence of densities invokes the Radon–Nikodym Theorem. The CDF connects to Measure Theory through Lebesgue–Stieltjes measures, and transformations $g (X)$ link to the Change of Variables formula. Conditioning generalizes to Conditional Expectation as a projection onto a sub- $σ$ -algebra.

Real-World Applications

Statistical inference: estimators (sample mean, variance) are random variables whose distributions justify confidence intervals and hypothesis tests.
Quantitative finance: the Black–Scholes model treats log-returns as normal random variables; option prices are expectations under a risk-neutral measure.
Statistical mechanics: the energy of a system in the canonical ensemble is a random variable with a Boltzmann distribution; macroscopic averages are expectations.
Information theory (Shannon): entropy $H (X) = - \sum p (x) lo g p (x)$ is a functional of a discrete random variable, bounding lossless compression.
Machine learning: loss functions are expectations over data-generating random variables; stochastic gradient descent samples the gradient as a random variable.
Reliability engineering: time-to-failure is modeled as an exponential or Weibull random variable, giving mean-time-between-failures.
Queueing & telecom: Poisson random variables count packet arrivals; service times are exponential random variables in M/M/1 queues.

Historical Development

1654 — Pascal and Fermat correspond on the "problem of points," computing expected stakes without naming random variables.
1713 — Jacob Bernoulli's Ars Conjectandi proves the first law of large numbers for binomial counts.
1733/1812 — De Moivre and Laplace develop the normal approximation; Laplace's Théorie analytique systematizes expectations and generating functions.
1867 — Chebyshev introduces the inequality bearing his name, treating moments of "variables" rigorously and proving a general LLN.
1901–1909 — Lyapunov and Markov sharpen the Central Limit Theorem using characteristic functions of sums of random variables.
1902–1904 — Lebesgue's integration theory provides the analytic engine for defining expectation as an integral.
1909–1933 — Borel and Fréchet develop measure on abstract spaces, paving the way for measurable functions.
1933 — Kolmogorov's Grundbegriffe gives the modern axiomatic definition: a random variable is a measurable function on a probability space.
1930s–40s — Doob and Kolmogorov extend the concept to stochastic processes (families of random variables indexed by time).

Subtleties & Limits

Beyond the discrete/continuous dichotomy. Many distributions are neither purely discrete nor absolutely continuous. By the Lebesgue decomposition, every distribution splits into a discrete part, an absolutely continuous part, and a singular continuous part (e.g., the Cantor distribution), which has a continuous CDF but no density and assigns probability to a set of Lebesgue measure zero. Undergraduate courses usually ignore the singular part, yet it appears in dynamical systems and fractal models. Open practical questions: how to estimate and simulate from singular distributions efficiently, and how to define meaningful "densities" via Hausdorff measures. More foundationally, in infinite-dimensional settings (e.g., random functions, Gaussian processes) there is no Lebesgue measure, so the Radon–Nikodym viewpoint of equation [4] must be replaced by abstract measure theory — a frontier where measurability, the seemingly pedantic condition [2], becomes genuinely delicate.