Cumulative Distribution Functions and Change of Variables

The CDF and its change-of-variables machinery are the working core of computational physics. Every Monte Carlo simulation — lattice QCD, radiation transport in reactor and medical dosimetry codes, path-integral sampling in condensed matter — rests on turning uniform pseudo-random numbers into physical samples, by inverse-transform sampling where the CDF inverts cleanly and by Box–Muller or the Ziggurat algorithm for Gaussians where it does not. Error propagation leans on the Jacobian rule directly: when a measured period $T$ is reported as a frequency $\nu=1/T$, the exact transformed density comes from $\lvert\mathrm d T/\mathrm d\nu\rvert$, and the linearised $\sigma_\nu\approx\lvert g'\rvert\sigma_T$ is just its first-order shadow. In statistical mechanics the density of states is a change of variables from a uniform-in-phase-space measure to energy. And across data science the probability integral transform underlies quantile normalisation, copula models of dependence, and the goodness-of-fit tests (Kolmogorov–Smirnov, Anderson–Darling) that compare an empirical CDF to a theoretical one — the same $F_X$, now estimated from data.

Think of the CDF as accumulated probability mass swept in from the left. At a point mass it jumps by the size of that mass; over a smooth stretch it climbs at a rate equal to the density; the total height climbed from $-\infty$ to $+\infty$ is exactly $1$. Because it is built from the probabilities of nested events $\{X\le x\}$, the CDF exists even when no density does — a discrete variable has a staircase $F$ with no derivative at its steps, yet $F$ still tells you everything. That is why every clean derivation about transformations starts with $F$ and only differentiates at the very end. The first figure shows the two faces at once: the bell of a density and the S-curve of its integral, tied together by the fact that the shaded area to the left of any threshold is the height of $F$ there.

The Jacobian factor is bookkeeping for stretching. A map $g$ that compresses a small interval crowds the same probability into less length, so the density must rise by $\lvert\mathrm{d}x/\mathrm{d}y\rvert$; a map that stretches thins it out. Probability — not density — is the invariant, and the modulus is the guarantee that a fold or a reflection never flips the sign of a positive measure. The most operational form of this idea is inverse-transform sampling: feed a uniform number $u\in(0,1)$ into $F_X^{-1}$ and it lands exactly where the density is large, because the inverse CDF spends more horizontal room where $F$ rises slowly. The second figure makes that concrete with a real distribution and a real number — the median of an exponential comes out where it must.

The starting definition is a probability of an event, nothing more: the CDF is the chance that $X$ does not exceed a threshold. Everything structural about it follows from the axioms applied to the nested events $\{X\le x\}$.

where: $F_X$ — cumulative distribution function of $X$; $x$ — real threshold value; $P(\cdot)$ — probability measure on events. Reading: $F_X(x)$ is the accumulated probability to the left of and including $x$, and it is defined for every random variable whatsoever. Three properties characterise a CDF completely — any function with them is the CDF of some random variable. They are consequences of monotonicity and continuity of the measure, derived below.

where: $F_X(x^+)$ — right-hand limit, so $F_X$ is right-continuous; non-decreasing — from event inclusion $\{X\le x_1\}\subseteq\{X\le x_2\}$ for $x_1\le x_2$. Reading: jumps are allowed (point masses), plateaus are allowed (gaps in support), but $F$ never falls and always tops out at $1$. Two thresholds give an interval probability by subtraction, and differentiating $F$ where it is smooth recovers the density — the density is the local rate of accumulation.

where: $f_X$ — probability density, exists where $F_X$ is differentiable; $a,b$ — interval endpoints with $a<b$. Reading: $f_X\,\mathrm{d}x$ is the probability in an infinitesimal window, so $f_X$ is an integrand, which is why transforming it obeys the integral change-of-variables rule. For a strictly monotone, differentiable map the CDF composes directly and the density picks up the reciprocal slope in modulus — the one-dimensional Jacobian.

where: $Y=g(X)$ — transformed variable, $g$ strictly monotone and differentiable; $g^{-1}$ — inverse map, $x=g^{-1}(y)$; $\lvert\mathrm{d}x/\mathrm{d}y\rvert$ — absolute reciprocal slope. Reading: compress by $g$ and the density rises; stretch and it falls; the modulus keeps it positive for decreasing maps. In $n$ dimensions the reciprocal slope becomes the absolute value of the Jacobian determinant of the inverse map — the exact factor from multivariable calculus.

where: $\mathbf X,\mathbf Y$ — $n$-dimensional random vectors with $\mathbf Y=\mathbf g(\mathbf X)$; $\partial\mathbf x/\partial\mathbf y$ — Jacobian matrix of $\mathbf x=\mathbf g^{-1}(\mathbf y)$; $\det$ — determinant. Reading: the determinant is the local volume-scaling of the map, and the absolute value preserves positivity of the measure. When $g$ is not one-to-one, every real preimage of $y$ contributes, and their densities are added, each weighted by its own local slope.

where: $x_k$ — the $k$-th real root of $g(x)=y$; $g'(x_k)$ — slope of $g$ there. Reading: a fold such as $Y=X^2$ gathers mass from both $\pm\sqrt{y}$; a vanishing slope on any branch makes the density diverge. Feeding a variable through its own CDF flattens it to a uniform, and inverting that statement turns a uniform generator into a sampler for any target.

where: $U$ — uniform random variable on $(0,1)$; $F_X^{-1}$ — quantile function, the generalised inverse of $F_X$. Reading: this is the probability integral transform, the reason a single stream of uniforms can simulate any continuous distribution. Finally, the density of a sum of independent variables is the convolution of their densities — a two-dimensional Jacobian rule with the marginal integrated out.

where: $Z=X+Y$ — sum of independent random variables; the integral is the convolution $f_X*f_Y$. Reading: convolution is why sums smooth out and, iterated, drift toward the Gaussian — the thread picked up in aq-mp-laws-of-large-numbers-convergence.

1 · Standardising a Gaussian is an affine change of variables (confidence builder)

A calibration voltage reads $X\sim\mathcal N(\mu=5.00\ \mathrm V,\ \sigma=0.20\ \mathrm V)$. Show that the standardising map $Y=(X-\mu)/\sigma$ produces a standard normal, then find $P(X\le 5.30\ \mathrm V)$.

Step 1. The map is affine, $Y=g(X)=(X-\mu)/\sigma$ with inverse $x=\sigma y+\mu$ and $\mathrm{d}x/\mathrm{d}y=\sigma$. The linear-map rule (equation 4) gives $f_Y(y)=\sigma\,f_X(\sigma y+\mu)$. The density carries a factor $\lvert a\rvert=\sigma$ because the map stretches by $\sigma$.

Step 2. Substitute the Gaussian $f_X(x)=\tfrac{1}{\sigma\sqrt{2\pi}}e^{-(x-\mu)^2/2\sigma^2}$: the argument $\sigma y+\mu$ makes $(x-\mu)^2/2\sigma^2=y^2/2$, and the prefactor $\sigma\cdot\tfrac{1}{\sigma\sqrt{2\pi}}=\tfrac{1}{\sqrt{2\pi}}$, so $f_Y(y)=\tfrac{1}{\sqrt{2\pi}}e^{-y^2/2}$. Y is exactly standard normal, $\mathcal N(0,1)$ — the affine map has erased $\mu$ and $\sigma$.

Step 3. Because $\{X\le 5.30\}=\{Y\le(5.30-5.00)/0.20\}=\{Y\le 1.5\}$, the CDF transforms by composition: $F_X(5.30)=\Phi(1.5)$. From the standard normal table $\Phi(1.5)=\mathbf{0.9332}$. So $P(X\le 5.30\ \mathrm V)=0.9332$, i.e. about $93.3\%$ of readings fall at or below $5.30\ \mathrm V$. The whole computation is one change of variables plus one table lookup.

2 · Inverse-transform sampling of radioactive lifetimes (fully worked)

Decay times of an unstable nucleus follow $X\sim\mathrm{Exp}(\lambda)$ with $\lambda=0.5\ \mathrm{s^{-1}}$ (mean lifetime $\tau=1/\lambda=2\ \mathrm s$). A random-number generator delivers uniforms $U=0.1,\ 0.5,\ 0.9$. Convert them to decay times and check the median.

Step 1. Write the target CDF by integrating the density from $0$: $F_X(x)=1-e^{-\lambda x}$ for $x\ge0$. It is strictly increasing, so the inverse is unique. This is the exact setting of the probability integral transform.

Step 2. Invert: $u=1-e^{-\lambda x}\Rightarrow x=F_X^{-1}(u)=-\tfrac1\lambda\ln(1-u)$. With $\lambda=0.5$, $X=-2\ln(1-U)$. Feeding a uniform through the inverse CDF is the change of variables $x=F_X^{-1}(u)$ whose Jacobian cancels the target density.

Step 3. Evaluate at the three uniforms: $U=0.1\Rightarrow X=-2\ln 0.9=\mathbf{0.2107\ s}$; $U=0.5\Rightarrow X=-2\ln 0.5=\mathbf{1.3863\ s}$; $U=0.9\Rightarrow X=-2\ln 0.1=\mathbf{4.6052\ s}$. The samples crowd toward small $x$, exactly as the second figure shows, because the exponential density is largest there.

Step 4. Check the median: $U=0.5$ gave $X=1.3863\ \mathrm s$, and the true median solves $F_X(m)=0.5$, i.e. $m=\ln 2/\lambda=0.6931/0.5=\mathbf{1.3863\ s}$ — they agree. Confirming the distribution: $P(X\le 1.3863)=1-e^{-0.5\times1.3863}=1-e^{-0.6931}=1-0.5=\mathbf{0.5}$, so the sampler reproduces $\mathrm{Exp}(\lambda)$ as required. Since $1-U$ is uniform whenever $U$ is, $X=-\tfrac1\lambda\ln U$ works equally and saves one subtraction — the form used in production Monte Carlo codes. As a cross-check of the folding rule, the derivation above gives $Y=X^2\sim\chi^2_1$ with $f_Y(1)=\tfrac{1}{\sqrt{2\pi}}e^{-1/2}=\mathbf{0.2420}$ and an integrable spike as $y\to0^+$.

The change-of-variables factor that runs through this whole article is one man's determinant: when a distribution is pushed through a map, the density is rescaled by the absolute value of the Jacobian — Carl Gustav Jacob Jacobi's functional determinant, introduced in a memoir whose subject was exactly how integrands transform under a change of variables.

Texts. Stone and Goldbart, Mathematics for Physics (CUP, 2009), probability chapter · Arfken, Weber and Harris, Mathematical Methods for Physicists, 7th ed., Ch. 23 · Riley, Hobson and Bence, Mathematical Methods for Physics and Engineering, 3rd ed., Ch. 30 · Grimmett and Stirzaker, Probability and Random Processes, 3rd ed. · Feller, An Introduction to Probability Theory and Its Applications, Vol. II · Press et al., Numerical Recipes, 3rd ed., Ch. 7 (random deviates). The three defining properties of $F_X$ are derived from the axioms (monotonicity and continuity of the measure); the change-of-variables rule is Jacobi's functional determinant applied to a normalised integrand. Key numerical anchors: (1) the $\chi^2_1$ density $f_Y(y)=1/\sqrt{2\pi y}\,e^{-y/2}$ evaluates to $0.2420$ at $y=1$, with normalisation confirmed via the substitution $y=t^2$; (2) the Box–Muller Jacobian gives $\lvert\det\partial(u_1,u_2)/\partial(z_1,z_2)\rvert=\tfrac{1}{2\pi}e^{-(z_1^2+z_2^2)/2}$; (3) the exponential sampler $X=-\tfrac1\lambda\ln(1-U)$ reproduces $F_X=1-e^{-\lambda x}$ and the median $\ln2/\lambda$; (4) the standardising affine map returns $\Phi(1.5)=0.9332$.

Prerequisites. Axioms of Probability · Random Variables and Probability Distributions · Joint, Marginal and Conditional Distributions. Leads to. Gaussian (Normal) Distribution · Multivariate Gaussian Distribution · Moment-Generating and Characteristic Functions · Laws of Large Numbers and Convergence of Random Variables.

Ask the tutor: Bring me any transformation $Y=g(X)$ — a square, a log, or a distribution you want to sample — and we will push it through the CDF and read off the Jacobian together.