Random Variables and Probability Distributions

A random variable is the hinge that turns the abstract machinery of probability into ordinary calculus. Probability theory begins austere: a sample space $\Omega$ of elementary outcomes, a $\sigma$ -algebra $\mathcal{F}$ of events, and a measure $P$ obeying the axioms. Nothing in that triple mentions numbers. A random variable supplies them — it is a function $X:\Omega\to\mathbb{R}$ that reads a real number off each outcome, subject to one technical demand: the preimage of every half-line must be an event, $\{\omega:X(\omega)\le x\}\in\mathcal{F}$ , so that $\Pr(X\le x)$ always makes sense. That single condition, measurability, is what lets us forget $\Omega$ entirely and work on the real line. Once $X$ is in hand, all its probabilistic content is the push-forward measure: how $P$ redistributes itself along $\mathbb{R}$ . The cleanest packaging of that measure is the cumulative distribution function $F_X(x)=\Pr(X\le x)$ — a non-decreasing, right-continuous climb from $0$ to $1$ that every random variable possesses. From its shape the two familiar species emerge. A staircase $F_X$ is a discrete variable, described by a probability mass function; an absolutely continuous $F_X$ is a continuous variable, described by a density $f_X=F_X'$ . Mixed and singular cases exist too, which is exactly why the CDF, not the PMF or PDF, is the fundamental object. This is where statistical physics, quantum measurement, and error analysis all plug in. The Boltzmann factor is a density over energies; $|\psi|^2$ is a density over positions; detector counts obey a mass function. Master the random variable, and the named families — Bernoulli, Poisson, Gaussian — and the Central Limit Theorem become variations on one theme.

Key takeaways

A random variable $X$ is a measurable map $X:\Omega\to\mathbb{R}$ : it labels each outcome with a number so that the preimage of every interval is an event with a well-defined probability.
Its distribution is captured completely by the cumulative distribution function $F_X(x)=\Pr(X\le x)$ — non-decreasing, right-continuous, rising from $0$ to $1$ .
Discrete variables carry a probability mass function $p_X(x_i)$ that sums to $1$ ; continuous ones carry a density $f_X(x)=F_X'(x)$ that integrates to $1$ .
A density is not a probability: $f_X(x)$ has units of inverse length, can exceed $1$ , and $\Pr(X=x)=0$ for a continuous variable. Only $f_X\,\mathrm{d}x$ and integrals mean anything.
The CDF is universal — it exists for discrete, continuous, and mixed variables alike — and it drives change-of-variables, sampling, and every downstream summary.
Key idea: a random variable is a measurable map $X:\Omega\to\mathbb{R}$ ; its entire probabilistic content is the push-forward measure on $\mathbb{R}$ , summarized universally by the CDF $F_X(x)=\Pr(X\le x)$ and specialized to a PMF (discrete) or a PDF (continuous).

Where this lives today

The random variable is the load-bearing abstraction of computational physics. Every Monte Carlo simulation — lattice QCD sampling gauge-field configurations, statistical-mechanics codes exploring a canonical ensemble, radiation-transport codes tracing neutrons — is machinery for drawing random variables from a target distribution, and inverse-transform and rejection sampling both run on the CDF exactly as in Example 2. At the LHC, event generators synthesize billions of collisions by pushing pseudo-random uniforms through the distributions of decay times and momenta, then compare the histograms against detector counts that are themselves Poisson random variables. In quantum mechanics the Born rule is the statement that a measurement outcome is a random variable whose density is $|\psi(x)|^2$ , with normalization $\int|\psi|^2=1$ the density condition (5). The open frontier is high-dimensional sampling: drawing from distributions over millions of variables, where Hamiltonian Monte Carlo and diffusion-model samplers are actively reshaping how physics simulations are done.

The physical picture

Think of $X$ as a measuring instrument wired to the universe. Nature selects an outcome $\omega$ — a coin lands, a nucleus decays, a photon strikes a detector — and the instrument reports a number $X(\omega)$ . The randomness lives entirely in the choice of $\omega$ ; the rule $X$ itself is fixed and deterministic. The distribution of $X$ answers the only question that survives this reading: how often does the needle land in each range? The map from $\Omega$ to $\mathbb{R}$ is a deliberate forgetting — two utterly different physical setups can push forward to the same distribution, which is precisely why a handful of named laws describe enormous swaths of physics. The cumulative distribution function is a running tally. Sweep the threshold $x$ from $-\infty$ to $+\infty$ and $F_X(x)$ accumulates every scrap of probability seen so far, climbing monotonically from $0$ to $1$ . A vertical jump is an atom — a single value carrying positive probability, the fingerprint of a discrete variable. A smooth rise is spread-out probability, and its local slope is the density. Because both features live inside one function, the CDF is the single descriptor that works for every random variable, and reading off jumps versus slopes recovers the PMF versus the PDF.

Figure

The CDF is the one universal descriptor. Left: a discrete variable (Bernoulli, p = 0.6) has a staircase CDF whose two jump heights, 0.4 and 0.6, are exactly the probability masses. Right: a continuous variable has a smooth CDF whose local slope is the density f_X = dF_X/dx. Reading off jumps gives the PMF; reading off slopes gives the PDF.

The density itself is best pictured as mass smeared along a wire of unit total mass. Asking the density at a point is like asking the mass-per-length there; asking the probability of one exact value is like asking the mass of a single dimensionless point — which is zero. Only segments of the wire carry weight. This is the source of the field's most persistent confusion: $f_X(x)$ is probability per unit length, so it can spike arbitrarily high near a sharp peak while every single point still has probability zero. The concrete distribution physicists reach for first is the exponential — the waiting-time law of radioactive decay and Poisson arrivals — and its density and CDF make the whole story quantitative.

Figure

The exponential distribution with rate λ = 1. The density f_X(x) = λe^(−λx) (amber, shaded) falls from λ at the origin; the CDF F_X(x) = 1 − e^(−λx) (blue) climbs from 0 to 1. Its median ln2/λ ≈ 0.693 sits to the left of its mean 1/λ = 1 — the signature right-skew — and the shaded area under f_X up to any x equals the height of F_X there.

The core equations

The definition is the entire subject in one line: a random variable is a function on outcomes, constrained so that thresholding it always yields an event. Without measurability the symbol $\Pr(X\le x)$ would be meaningless; with it, every question about $X$ is a legitimate probability.

X:\Omega\to\mathbb{R},\qquad \{\omega\in\Omega : X(\omega)\le x\}\in\mathcal{F}\ \ \forall x\in\mathbb{R} \qquad (1)

where: $\Omega$ — sample space of elementary outcomes; $\mathcal{F}$ — the $\sigma$ -algebra of events (measurable subsets of $\Omega$ ); $\omega$ — a single outcome; $X(\omega)$ — the real value the variable assigns to $\omega$ . Read it as: whatever threshold $x$ you pick, the set of outcomes landing at or below it is something $P$ can measure. Measurability is what lets us leave $\Omega$ behind. All the probability that $X$ carries is repackaged as a single function of the real threshold — the cumulative distribution function, the honest universal descriptor of the distribution.

F_X(x)=\Pr(X\le x)=P\big(\{\omega:X(\omega)\le x\}\big) \qquad (2)

where: $F_X$ — the CDF of $X$ ; $P$ — the probability measure on $(\Omega,\mathcal{F})$ ; $x$ — the threshold value on the real line. The value $F_X(x)$ is the total probability that has accumulated by the time the sweep reaches $x$ . Not every function can be a CDF. Four properties, forced directly by the probability axioms, characterize exactly which ones are — and Derivation A proves each from the continuity of the measure.

\lim_{x\to-\infty}F_X(x)=0,\quad \lim_{x\to+\infty}F_X(x)=1,\quad x_1\le x_2\Rightarrow F_X(x_1)\le F_X(x_2),\quad F_X(x^+)=F_X(x) \qquad (3)

where: the two limits fix the endpoints at $0$ and $1$ ; monotonicity encodes that accumulated probability never decreases; $F_X(x^+)$ — the right-hand limit, equal to $F_X(x)$ , so a CDF is right-continuous. The left limit need not agree: a gap $F_X(x)-F_X(x^-)$ is a jump, and it equals $\Pr(X=x)$ . When $F_X$ is a staircase, all the probability sits on a countable set of points, and the natural descriptor is the probability mass function — the jump heights themselves.

p_X(x_i)=\Pr(X=x_i)\ge 0,\qquad \sum_i p_X(x_i)=1 \qquad (4)

where: $p_X$ — the PMF, the probability of each attainable value; $x_i$ — the countable support points of $X$ . The masses are non-negative and exhaust the unit probability. Bernoulli, binomial, and Poisson variables all live here. When $F_X$ instead rises smoothly, its derivative is the probability density, and probabilities become integrals. The infinitesimal reading — $f_X(x)\,\mathrm{d}x$ is the probability of the sliver $(x,x+\mathrm{d}x]$ — is the one to keep in mind, because it exposes that a density carries units of inverse length.

f_X(x)=\frac{\mathrm{d}F_X}{\mathrm{d}x}\ge 0,\qquad \Pr(a<X\le b)=\int_a^b f_X(x)\,\mathrm{d}x,\qquad \int_{-\infty}^{\infty}f_X(x)\,\mathrm{d}x=1 \qquad (5)

where: $f_X$ — the PDF, probability per unit length, $[f_X]=[\text{length}]^{-1}$ ; $a,b$ — the interval endpoints. There is no constraint $f_X\le 1$ ; only integrals of $f_X$ are pinned to $[0,1]$ . Uniform, exponential, and Gaussian variables live here. Physics constantly relabels its variables — energy to momentum, position to angle — and the density must transform to keep total probability at one. The rule carries a Jacobian factor precisely because $f_X$ has units of inverse length; the CDF, being dimensionless accumulated probability, does not. Derivation B builds it from that conservation law.

f_Y(y)=f_X\big(g^{-1}(y)\big)\left|\frac{\mathrm{d}g^{-1}}{\mathrm{d}y}\right|,\qquad Y=g(X),\ g\ \text{strictly monotone} \qquad (6)

where: $g$ — a strictly monotone, differentiable map; $g^{-1}$ — its inverse; $|\mathrm{d}g^{-1}/\mathrm{d}y|$ — the Jacobian that stretches or compresses the density so probability is conserved and $f_Y\ge0$ . The absolute value covers both increasing and decreasing $g$ . Finally, to compress a distribution into a single number, one averages a function of $X$ against its law. The same formula, sum or integral, governs the mean, the variance, and every higher moment — the subject of the article this one leads into.

\mathbb{E}[h(X)]=\sum_i h(x_i)\,p_X(x_i)\quad\text{or}\quad \int_{-\infty}^{\infty} h(x)\,f_X(x)\,\mathrm{d}x \qquad (7)

where: $\mathbb{E}[\cdot]$ — the expectation operator; $h$ — any measurable function of $X$ . This is the law of the unconscious statistician: to average $h(X)$ you never need the distribution of $h(X)$ itself, only that of $X$ .

Derivation AThe four CDF properties from continuity of the measure

The goal is to prove that the four defining properties of a cumulative distribution function — the limits to zero and one, monotonicity, and right-continuity, together with the atom formula for jumps — are not conventions but forced consequences of the probability axioms applied to nested families of events. Each property is a statement about how the measure $P$ behaves along an increasing or decreasing chain of sets, and the tool throughout is the continuity of $P$ : the measure of a monotone limit of events is the limit of their measures. We take the four in turn and read each off the appropriate chain.

Step 1. For the endpoint limits, consider the events $A_x=\{X\le x\}$ . As $x\to-\infty$ these shrink to the empty set $\varnothing$ , and as $x\to+\infty$ they grow to fill $\Omega$ ; continuity of the measure then gives $F_X(x)\to P(\varnothing)=0$ and $F_X(x)\to P(\Omega)=1$ respectively. Why: a probability measure assigns $0$ to the impossible event and $1$ to the certain one, and continuity lets those endpoint values transfer to the limit of the CDF.

Step 2. For monotonicity, note that if $x_1\le x_2$ then $\{X\le x_1\}\subseteq\{X\le x_2\}$ , since any outcome at or below $x_1$ is automatically at or below $x_2$ . Monotonicity of $P$ on nested sets gives $F_X(x_1)\le F_X(x_2)$ immediately. Why: enlarging the threshold can only add outcomes to the tallied event, never remove them, so accumulated probability cannot decrease.

Step 3. For right-continuity, write $\{X\le x\}=\bigcap_{n\ge1}\{X\le x+\tfrac1n\}$ , a decreasing intersection of events. Continuity from above of $P$ yields $F_X(x+\tfrac1n)\to F_X(x)$ , which is exactly the statement $F_X(x^+)=F_X(x)$ . Why: the threshold $\le x$ is inclusive of the endpoint, so approaching from the right must recover the value at $x$ itself, not overshoot it.

Step 4. For the atom formula, approach from the left instead: $\{X<x\}=\bigcup_{n\ge1}\{X\le x-\tfrac1n\}$ gives the left limit $F_X(x^-)=\Pr(X<x)$ , so the jump is $F_X(x)-F_X(x^-)=\Pr(X\le x)-\Pr(X<x)=\Pr(X=x)$ . Why: the discrepancy between the inclusive and exclusive thresholds is precisely the probability sitting exactly at $x$ — the height of the staircase step. *Verified: reproduces the standard characterization in Grimmett & Stirzaker and Feller Vol. I; the jump equals $\Pr(X=x)$ and vanishes wherever $F_X$ is continuous.*

Go deeper · Derivation B · 1 minThe change-of-variables Jacobian from conservation of probability

Derivation BThe change-of-variables Jacobian from conservation of probability

The goal is to derive the transformation law for a probability density under a smooth, strictly monotone relabeling $Y=g(X)$ , and to see why a Jacobian factor is unavoidable in the density while entirely absent from the CDF relation. The physical principle driving the whole calculation is that probability is conserved under any invertible relabeling of the outcomes: the chance that $X$ lands in a set must equal the chance that $Y$ lands in the image of that set. We enforce this at the level of the CDF, where it is exact and factor-free, and then differentiate to descend to densities, where the stretching of the axis leaves its mark.

Step 1. Take $g$ strictly increasing first. Then $\{g(X)\le y\}=\{X\le g^{-1}(y)\}$ as events, so equating their probabilities gives $F_Y(y)=\Pr(g(X)\le y)=\Pr(X\le g^{-1}(y))=F_X(g^{-1}(y))$ . Why: for an increasing map, thresholding $Y$ at $y$ is identical to thresholding $X$ at $g^{-1}(y)$ , so the accumulated probabilities coincide with no correction whatsoever — the CDF simply composes with $g^{-1}$ .

Step 2. Differentiate this identity with respect to $y$ using the chain rule: $f_Y(y)=\dfrac{\mathrm{d}}{\mathrm{d}y}F_X(g^{-1}(y))=f_X(g^{-1}(y))\,\dfrac{\mathrm{d}g^{-1}}{\mathrm{d}y}$ , and for increasing $g$ the derivative $\mathrm{d}g^{-1}/\mathrm{d}y$ is positive. Why: the density is the slope of the CDF, so differentiating a composed function drags down the Jacobian $\mathrm{d}g^{-1}/\mathrm{d}y$ , which measures how much the $y$ -axis is stretched relative to the $x$ -axis.

Step 3. For strictly decreasing $g$ , the inequality flips: $\{g(X)\le y\}=\{X\ge g^{-1}(y)\}$ , so $F_Y(y)=1-F_X(g^{-1}(y))$ and differentiating gives $f_Y(y)=-f_X(g^{-1}(y))\,\mathrm{d}g^{-1}/\mathrm{d}y$ , where now $\mathrm{d}g^{-1}/\mathrm{d}y<0$ . Both cases collapse into one by taking the absolute value, $f_Y(y)=f_X(g^{-1}(y))\,|\mathrm{d}g^{-1}/\mathrm{d}y|$ . Why: a density can never be negative, and the absolute value is exactly the bookkeeping that keeps $f_Y\ge0$ while conserving total probability, regardless of whether the map preserves or reverses orientation. *Verified: matches the change-of-variables treatment in Riley-Hobson-Bence ch. 30 and Arfken & Weber; $\int f_Y\,\mathrm{d}y=\int f_X\,\mathrm{d}x=1$ by the substitution rule.*

Go deeper · Derivation C · 1 minThe exponential distribution from normalization to median

Derivation CThe exponential distribution from normalization to median

The goal is to take a single concrete continuous law — the exponential density $f_X(x)=\lambda e^{-\lambda x}$ for $x\ge0$ and zero otherwise, with rate $\lambda>0$ — and carry it through the full life cycle every distribution demands: confirm it is a legitimate density by normalizing it, integrate it into its CDF, and locate its median. The exponential is the canonical waiting-time law, governing the interval between radioactive decays or Poisson-distributed detector clicks, so pinning down these three quantities is the working physicist's first calculation with any new distribution.

Step 1. Normalize. Integrate the density over its support: $\int_0^{\infty}\lambda e^{-\lambda x}\,\mathrm{d}x=\big[-e^{-\lambda x}\big]_0^{\infty}=0-(-1)=1$ . Since $f_X\ge0$ everywhere and integrates to one, it is a valid density. Why: normalization is the non-negotiable membership test for a PDF — it is the continuous echo of "the probabilities sum to one" — and it fixes the prefactor $\lambda$ uniquely.

Step 2. Build the CDF by accumulating the density up to $x$ : $F_X(x)=\int_0^{x}\lambda e^{-\lambda t}\,\mathrm{d}t=\big[-e^{-\lambda t}\big]_0^{x}=1-e^{-\lambda x}$ for $x\ge0$ , and $0$ for $x<0$ . This rises from $0$ , is right-continuous and strictly increasing, and tends to $1$ — every CDF axiom satisfied. Why: the CDF is the running integral of the density, so this step both delivers the distribution's universal descriptor and re-verifies the four properties of Derivation A on a live example.

Step 3. Find the median $m$ by solving $F_X(m)=\tfrac12$ : $1-e^{-\lambda m}=\tfrac12\Rightarrow e^{-\lambda m}=\tfrac12\Rightarrow m=\dfrac{\ln 2}{\lambda}\approx\dfrac{0.6931}{\lambda}$ . The mean, by contrast, is $\mathbb{E}[X]=\int_0^\infty x\,\lambda e^{-\lambda x}\,\mathrm{d}x=1/\lambda$ , strictly larger than the median. Why: the median splits the probability into two equal halves while the mean is pulled rightward by the long exponential tail — the gap $1/\lambda>\ln2/\lambda$ is the quantitative signature of the distribution's right skew. *Verified: mean $=1/\lambda$ , median $=\ln 2/\lambda\approx0.6931/\lambda$ , CDF $=1-e^{-\lambda x}$ — all standard textbook values, reconfirmed numerically for $\lambda=1$ (median $0.6931$ , mean $1$ ).*

Limiting cases

The CDF is a single object that specializes to every familiar form according to its regularity — the reason it, and not the PMF or PDF, deserves the title of fundamental descriptor.

Case	Result	Meaning
Purely discrete	$F_X(x)=\sum_{x_i\le x} p_X(x_i),\ \sum_i p_X(x_i)=1$	Staircase CDF; $X$ takes a countable set of values with masses $p_X(x_i)=\Pr(X=x_i)$ . Bernoulli, binomial, Poisson.
Purely continuous	$F_X(x)=\int_{-\infty}^{x} f_X(t)\,\mathrm{d}t,\ \int_{-\infty}^{\infty} f_X=1$	Absolutely continuous CDF with density $f_X=F_X'$ where the derivative exists. Uniform, exponential, Gaussian.
Mixed	$F_X=\alpha\,F^{\text{disc}}+(1-\alpha)\,F^{\text{cont}},\ 0<\alpha<1$	Jumps and smooth growth combined — e.g. a censored measurement piling probability at a boundary plus a continuous spread inside.
Degenerate (constant)	$\Pr(X=c)=1,\ F_X(x)=\Theta(x-c)$	The zero-variance limit: a sure value. The CDF is a single Heaviside step — the endpoint of the laws of large numbers.

Common trapA density is a probability

The single most common error in the whole subject is treating $f_X(x)$ as a probability. It is not. A density is probability per unit length: it carries units of inverse length, it can exceed $1$ (a uniform variable on the interval $(0,\tfrac12)$ has $f_X=2$ throughout, yet integrates to one), and for a continuous variable the probability of any exact single value is $\Pr(X=x)=0$ . Only $f_X(x)\,\mathrm{d}x$ , or an integral $\int_a^b f_X$ over a set, is a genuine probability. The mirror-image mistake is writing $\int p_X$ for a discrete variable — there you must sum the mass function, never integrate it. Densities integrate; masses sum; only the CDF is safe to read directly as a probability.

Worked examples

1 · Waiting time for a radioactive decay (fully worked)

A radioactive source decays at rate $\lambda=2\ \text{s}^{-1}$ , so the waiting time $X$ to the next decay is exponential with that rate. Find the mean and median waiting times, the probability of waiting longer than one second, and the probability of a decay within the first half-second.

Step 1. The mean waiting time is the reciprocal of the rate: $\mathbb{E}[X]=1/\lambda=1/2=\mathbf{0.5\ s}$ . This is the "expected" interval between clicks, but because the distribution is skewed it is not the fifty-fifty point.

Step 2. The median is $m=\ln 2/\lambda=0.6931/2=\mathbf{0.347\ s}$ . Half of all waiting intervals are shorter than $0.347$ s even though the mean is $0.5$ s — the long tail of occasional long waits drags the mean above the median, the right-skew made numerical.

Step 3. The probability of waiting longer than one second is the survival function $\Pr(X>1)=1-F_X(1)=e^{-\lambda\cdot1}=e^{-2}=\mathbf{0.135}$ . Roughly one interval in seven exceeds a full second.

Step 4. The probability of a decay within the first half-second is $\Pr(X\le 0.5)=F_X(0.5)=1-e^{-\lambda\cdot0.5}=1-e^{-1}=\mathbf{0.632}$ . Notice this is just above one-half, consistent with $0.5$ s lying just past the median $0.347$ s. Mean 0.5 s, median 0.347 s, Pr(X>1)=0.135, Pr(X≤0.5)=0.632.

2 · Inverse-transform sampling (change of variables)

A computer supplies $U$ uniform on $(0,1)$ , with density $f_U(u)=1$ there. Show that $X=-\tfrac{1}{\lambda}\ln(1-U)$ is exponential with rate $\lambda$ — the trick behind Monte Carlo event generators.

Step 1. Identify the map and invert it. Here $X=g(U)=-\tfrac{1}{\lambda}\ln(1-U)$ is strictly increasing on $(0,1)$ , and solving for $U$ gives the inverse $U=g^{-1}(X)=1-e^{-\lambda X}$ — which is exactly the exponential CDF, no accident, since inverse-transform sampling is built from $F_X^{-1}$ .

Step 2. Differentiate the inverse to get the Jacobian: $\dfrac{\mathrm{d}g^{-1}}{\mathrm{d}x}=\dfrac{\mathrm{d}}{\mathrm{d}x}\big(1-e^{-\lambda x}\big)=\lambda e^{-\lambda x}$ , which is positive for all $x\ge0$ , as it must be for an increasing map.

Step 3. Apply the transformation law (6): $f_X(x)=f_U\big(g^{-1}(x)\big)\,\big|\mathrm{d}g^{-1}/\mathrm{d}x\big|=1\cdot\lambda e^{-\lambda x}=\lambda e^{-\lambda x}$ for $x\ge0$ . The uniform density is $1$ , so the entire shape of the output distribution comes from the Jacobian. X is exponential(λ): f_X(x)=λe^(−λx). This is why feeding a stream of uniform random numbers through $F_X^{-1}$ synthesizes samples from any target distribution whose CDF you can invert.

Physics of the portrait

The random variable as we use it — a measurable function whose distribution is a push-forward measure — is barely ninety years old, even though physicists computed with probabilities for three centuries before it. The clean definition, and the axioms that make it rigorous, are the work of one mathematician who set out to give probability the same footing as geometry.

Origins · 1 minAndrey Kolmogorov and how this was discovered

Andrey Kolmogorov

1903–1987 · measure-theoretic foundations of probability

For two and a half centuries probability was a collection of brilliant but ad hoc calculations — Huygens on fair games, Bernoulli on the law of large numbers, Laplace on the whole classical edifice — without an agreed definition of what a probability was. Paradoxes multiplied whenever the sample space became infinite, and Hilbert listed the axiomatization of probability among his famous problems in 1900. The resolution came from Andrey Nikolaevich Kolmogorov, working in Moscow at the height of a school that had just made Lebesgue's measure theory its own. In his 1933 monograph Grundbegriffe der Wahrscheinlichkeitsrechnung — "Foundations of the Theory of Probability" — Kolmogorov identified a probability with a normalized measure on a $\sigma$ -algebra of events, and, decisively for this article, defined a random variable as a measurable function from the sample space to the reals. At a stroke the distribution became a push-forward measure, the CDF its universal fingerprint, and the whole apparatus of Lebesgue integration became available for expectations and limit theorems. Kolmogorov's reach extended far beyond foundations — turbulence, dynamical systems, algorithmic complexity, and the $K$ in KAM theory all bear his name — but it is the 1933 axioms that let a physicist write $\Pr(X\le x)$ and mean something exact. Every distribution in this chapter is a measure in his sense.

History

1657

— Huygens prints De ratiociniis in ludo aleae, the first treatise on probability

1713

— Jacob Bernoulli's Ars Conjectandi states the law of large numbers

1812

— Laplace's Théorie analytique des probabilités systematizes classical probability

1902

— Lebesgue's measure and integral supply the analytic backbone probability will need

1933

— Kolmogorov's Grundbegriffe axiomatizes probability and defines the random variable as a measurable function

Going further

Texts. Stone & Goldbart, Mathematics for Physics, ch. on probability and statistics · Arfken & Weber, Mathematical Methods for Physicists, ch. on probability · Riley, Hobson & Bence, Mathematical Methods for Physics and Engineering, ch. 30 (Probability) · Feller, An Introduction to Probability Theory and Its Applications, Vol. I · Grimmett & Stirzaker, Probability and Random Processes · Kolmogorov, Foundations of the Theory of Probability (1933, English 1956) for the axioms themselves.

Provenance. All CDF, PMF, and PDF properties are standard (Stone & Goldbart; Arfken & Weber; Riley-Hobson-Bence ch. 30; Grimmett & Stirzaker; Feller Vol. I). A verifier should recompute: (1) the four CDF properties from continuity of the measure, with jump $=\Pr(X=x)=F_X(x)-F_X(x^-)$ ; (2) the change-of-variables law $f_Y=f_X(g^{-1})\,|\mathrm{d}g^{-1}/\mathrm{d}y|$ from $F_Y(y)=F_X(g^{-1}(y))$ ; (3) exponential normalization $\int_0^\infty\lambda e^{-\lambda x}\,\mathrm{d}x=1$ , CDF $1-e^{-\lambda x}$ , mean $1/\lambda$ , median $\ln2/\lambda\approx0.6931/\lambda$ ; (4) the worked decay numbers at $\lambda=2\ \text{s}^{-1}$ : mean $0.5$ s, median $0.347$ s, $\Pr(X>1)=e^{-2}=0.135$ , $\Pr(X\le0.5)=1-e^{-1}=0.632$ ; (5) inverse-transform sampling reproducing $f_X=\lambda e^{-\lambda x}$ from a uniform seed. No nonstandard constants are asserted.

Prerequisites. Axioms of Probability · Conditional Probability and Bayes' Theorem. Leads to. Cumulative Distribution Functions and Change of Variables · Expectation, Variance and Moments · Gaussian (Normal) Distribution · Central Limit Theorem.

Ask the tutor: Want to see how the CDF you just met turns a stream of uniform random numbers into a Gaussian, or how it seeds a Monte Carlo simulation? Ask me to walk through inverse-transform sampling or the change-of-variables recipe step by step.

What is the difference between a random variable and its distribution?

The random variable $X$ is the function $X:\Omega\to\mathbb{R}$ — the rule that reads a number off each outcome. Its distribution is the pattern of how often those numbers fall in each range, formally the push-forward measure summarized by the CDF $F_X$ . Two different random variables, even built on different sample spaces, can share one distribution: a coin-flip counter and a dice-parity indicator are both Bernoulli. Physics almost always cares only about the distribution, which is why we can forget $\Omega$ once measurability guarantees the CDF exists.

Why insist on the measurability condition — does anything really go wrong without it?

Without measurability the symbol $\Pr(X\le x)$ can be genuinely undefined. For finite or countable sample spaces every subset is an event and the condition is automatic, so it looks like pedantry. But on continuous spaces there exist non-measurable sets — pathological subsets of the reals with no consistent probability — and a function landing you inside one would make $\{X\le x\}$ a set to which $P$ assigns no value. Requiring the preimage of every half-line to be an event rules this out and guarantees the CDF is well-defined everywhere. It is the minimal price for doing calculus with probability rigorously.

If Pr(X = x) = 0 for every point, how can X take any value at all?

This feels paradoxical but is not. For a continuous variable, probability is spread over intervals, not concentrated on points — like mass smeared along a wire, where any single dimensionless point has zero mass yet every segment has positive mass. When $X$ is realized it does land on some exact value, but the probability was always a statement about neighborhoods: $\Pr(x<X\le x+\mathrm{d}x)=f_X(x)\,\mathrm{d}x$ is positive for any finite sliver. The individual outcome has probability zero the same way a specific real number drawn uniformly from $(0,1)$ does — improbable in advance, certain to be something after the fact.

Why is the CDF, rather than the PDF or PMF, called the fundamental descriptor?

Because the CDF $F_X(x)=\Pr(X\le x)$ exists for every random variable without exception — discrete, continuous, mixed, or even the exotic singular-continuous case. A PMF requires the variable to live on a countable set; a PDF requires the distribution to be absolutely continuous. Neither exists in general, and a mixed variable (jumps plus a smooth part) has neither a pure PMF nor a pure PDF, yet its CDF is perfectly ordinary. The PMF is the set of jump heights of $F_X$ and the PDF is its derivative — both are special-case readings of the one object that always works.

Why does a Jacobian appear when I change variables in a density?

Because a density has units of inverse length, so it must rescale when you stretch the axis. Picture probability as conserved paint: relabeling $X$ to $Y=g(X)$ spreads the same total paint over a differently stretched line. Where $g$ expands a region, the same probability covers more length, so the density drops; where $g$ compresses, it rises. The factor $|\mathrm{d}g^{-1}/\mathrm{d}y|$ is exactly the local stretch ratio that keeps the paint conserved. The CDF needs no such factor because it is dimensionless accumulated probability — it simply composes with $g^{-1}$ , which is why deriving the rule from the CDF is the clean route.

How does this connect to expectation, moments, and the Gaussian?

Everything downstream is built on the distribution defined here. Summarizing a distribution by numbers — mean, variance, higher moments — is averaging powers of $X$ against its law via equation (7), the subject of the expectation-and-moments article. The named families (Bernoulli, Poisson, Gaussian) are simply distributions that recur in nature, and the Central Limit Theorem explains why sums of many independent random variables converge to the Gaussian regardless of their individual distributions. Each takes the CDF-and-density framework of this article as its starting point.