Moment-Generating and Characteristic Functions

Handed a probability distribution, a physicist wants one object that stores everything about it — every moment, how it behaves when independent copies pile up, and enough to reconstruct the distribution itself. Two closely related integral transforms do exactly this. The moment-generating function is the Laplace-type expectation $M_X(t)=\mathbb{E}[e^{tX}]$ ; the characteristic function is the Fourier-type expectation $\varphi_X(t)=\mathbb{E}[e^{itX}]$ . Both are the expectation operator acting on a one-parameter family of test weights, $e^{tx}$ or $e^{itx}$ , and both turn hard questions about distributions into easy questions about a single smooth function of $t$ . Three facts organize everything that follows. First, these transforms generate moments: expand $e^{tX}=\sum_n t^nX^n/n!$ , take expectations, and the $n$ -th derivative at the origin hands you $\mathbb{E}[X^n]$ . Second, they linearize convolution: for independent variables the transform of a sum is the product of transforms, so summing $n$ variables becomes multiplying $n$ functions instead of nesting $n$ integrals. Third, the characteristic function is universal and faithful: because $|e^{itx}|=1$ it converges for every distribution — even heavy-tailed laws like the Cauchy, where the MGF diverges — and by Fourier inversion it pins the distribution down uniquely. Take the logarithm and you get the cumulant-generating function, whose Taylor coefficients — the cumulants $\kappa_n$ — simply add over independent variables and vanish beyond second order precisely for the Gaussian. That single fact is why the characteristic function is the engine of the limit theorems: the asymptotics of a normalized sum follow from the local Taylor behaviour of $\varphi$ near $t=0$ , exactly where the low moments live. This article builds the definitions, the convergence subtleties, the inversion and uniqueness theorems, the transforms of the standard laws, and the characteristic-function proof of the Central Limit Theorem.

Key takeaways

The moment-generating function (MGF) $M_X(t)=\mathbb{E}[e^{tX}]$ stores all moments: its $n$ -th derivative at $t=0$ returns $\mathbb{E}[X^n]$ — for an Exponential of rate $2$ , this gives mean $0.5$ and variance $0.25$ .
The characteristic function (CF) $\varphi_X(t)=\mathbb{E}[e^{itX}]$ always exists (the integrand has modulus one), is the Fourier transform of the density, and by inversion determines the distribution uniquely.
Both transforms turn convolution into multiplication: for independent variables the transform of a sum is the product of transforms.
The logarithm is the cumulant-generating function $K_X(t)=\ln M_X(t)=\sum_n\kappa_n t^n/n!$ ; cumulants add over independent variables, with $\kappa_1=\mu$ , $\kappa_2=\sigma^2$ , and $\kappa_n=0$ for $n\ge3$ only for the Gaussian.
Lévy's continuity theorem promotes pointwise CF convergence into convergence in distribution, delivering the Central Limit Theorem from the one-line limit $(1-t^2/2n)^n\to e^{-t^2/2}$ .
The MGF can diverge (Cauchy, log-normal, any power-law tail) whereas the CF is universal — which is why probabilists reach for $\varphi_X$ .
Key idea: replace a distribution by the integral transform $\mathbb{E}[e^{tX}]$ or $\mathbb{E}[e^{itX}]$ — derivatives at the origin yield moments, the logarithm yields cumulants, products correspond to sums of independent variables, and inversion of the characteristic function recovers the distribution uniquely.

The core insight

Replace a probability distribution by the integral transform $\mathbb{E}[e^{tX}]$ or $\mathbb{E}[e^{itX}]$ : derivatives at the origin yield moments, the logarithm yields cumulants, products correspond to sums of independent variables, and (for the characteristic function) inversion recovers the distribution uniquely.

Where this lives today

The transform is a working tool across physics and its quantitative neighbours. In statistical mechanics the moment-generating function is the partition-function-like object $\mathbb{E}[e^{tX}]$ , its logarithm is a free energy, and the Legendre transform of the cumulant-generating function is the large-deviation rate function of the Gärtner–Ellis theorem — the object that governs exponentially rare fluctuations of currents, magnetizations, and energies. In particle and photon counting, the fact that independent Poisson streams add to a Poisson (Worked Example 3) underlies estimates of coincidence backgrounds and dark-count rates. In signal processing and quantitative finance the characteristic function is inverted numerically by fast Fourier transform to price options and fit heavy-tailed or Lévy-stable models where no MGF exists. Cumulants feed the Edgeworth expansions that correct the Gaussian approximation, and the saddle-point evaluation of an inverse transform — expanding $K_X(t)$ near its stationary point — is a standard asymptotic method in statistical physics. Open problems remain: characterizing which moment sequences determine a law (the moment problem) and controlling the rate of CLT convergence (Berry–Esseen and beyond) bear directly on how far a Gaussian approximation can be trusted.

The physical picture

Think of $e^{itx}$ as a continuum of unit-modulus weights indexed by the frequency $t$ . Averaging them over the distribution produces one complex number per $t$ ; the whole function $\varphi_X(t)$ is a fingerprint of the law. Small $t$ probes the bulk — the low moments — while large $t$ probes fine structure and the tails. Because the weights $e^{itx}$ never grow, the average is always well-defined: $\varphi_X(0)=1$ and $|\varphi_X(t)|\le1$ for every distribution, no matter how heavy its tail — the structural advantage over the moment-generating function, whose weights $e^{tx}$ blow up. The first figure shows the cleanest case, a law whose fingerprint is a copy of itself.

Figure

The characteristic function is the Fourier transform of the density. For a standard normal both are Gaussian bells: the density φ(x)=e^−x²/2/√(2π) peaks at 0.399, and its characteristic function φ(t)=e^−t²/2 peaks at 1 with φ(1)=0.607 and φ(2)=0.135 — bounded by 1 everywhere

The second fact is the one that makes limit theorems tractable. The density of a sum of independent variables is a convolution, and the Fourier transform of a convolution is a product — so summing $n$ variables is just raising a single function to the $n$ -th power. Watching what that repeated multiplication does to the shape of a distribution is the whole story of the Central Limit Theorem: convolving a law with itself and rescaling, over and over, grinds every bump smooth and drives the shape toward the one fixed point, the Gaussian. The second figure shows this collapse for the humblest starting law.

Figure

Central Limit Theorem as repeated convolution, all rescaled to variance 1. A single uniform is a flat box (height 0.289); the sum of two is a triangle (peak 0.408); the sum of many uniforms is indistinguishable from the standard normal (peak 0.399). Each uniform has variance 1/12, so twelve of them sum to N(6,1)

The core equations

The two definitions are a single idea — average an exponential weight — differing only in whether the exponent is real or imaginary. The real weight generates moments most transparently; the imaginary weight buys unconditional convergence and faithful inversion.

M_X(t)=\mathbb{E}\!\left[e^{tX}\right]=\int_{-\infty}^{\infty} e^{tx}\,f_X(x)\,\mathrm{d}x \qquad (1)

where: $M_X(t)$ — moment-generating function, defined only where the integral converges (some neighbourhood of $t=0$ , possibly just the point $t=0$ ); $f_X(x)$ — probability density of $X$ (replace the integral by $\sum_k e^{tx_k}p_k$ in the discrete case); $t$ — real transform variable. Reading: sweep the weight $e^{tx}$ across the distribution and record the average for each $t$ .

\varphi_X(t)=\mathbb{E}\!\left[e^{itX}\right]=\int_{-\infty}^{\infty} e^{itx}\,f_X(x)\,\mathrm{d}x \qquad (2)

where: $\varphi_X(t)$ — characteristic function; exists for all $t$ and every distribution because $|e^{itx}|=1$ , with $\varphi_X(0)=1$ and $|\varphi_X(t)|\le1$ ; $i$ — imaginary unit. This is literally the Fourier transform of the density, so all of Fourier analysis is available to probability. Differentiating either transform at the origin extracts moments; the imaginary unit in the CF version is absorbed by a factor $(-i)^n$ .

\mathbb{E}[X^n]=\left.\frac{\mathrm{d}^n M_X}{\mathrm{d}t^n}\right|_{t=0}=(-i)^n\left.\frac{\mathrm{d}^n \varphi_X}{\mathrm{d}t^n}\right|_{t=0} \qquad (3)

where: $\mathbb{E}[X^n]$ — $n$ -th raw moment; $(-i)^n$ — bookkeeping factor converting CF derivatives back to real moments. The transform is a generating function whose Taylor coefficients are the moments divided by factorials. The logarithm repackages moments into cumulants — the quantities that add over independent variables.

K_X(t)=\ln M_X(t)=\sum_{n=1}^{\infty}\frac{\kappa_n}{n!}\,t^n \qquad (4)

where: $K_X(t)$ — cumulant-generating function; $\kappa_n$ — $n$ -th cumulant, with $\kappa_1=\mu$ (mean), $\kappa_2=\sigma^2$ (variance), $\kappa_3=\mathbb{E}[(X-\mu)^3]$ (third central moment). Cumulants strip the combinatorial redundancy out of moments, so only the connected information survives. Independence turns the transform of a sum into a product — the property that makes the whole machine run.

X\perp Y\ \Rightarrow\ M_{X+Y}(t)=M_X(t)\,M_Y(t),\quad \varphi_{X+Y}(t)=\varphi_X(t)\,\varphi_Y(t) \qquad (5)

where: $\perp$ — statistical independence; $X+Y$ — sum whose density is the convolution $f_X*f_Y$ . Reading: multiplication of transforms is the convolution theorem in probabilistic dress; cumulants therefore add, $\kappa_n(X+Y)=\kappa_n(X)+\kappa_n(Y)$ . Scaling and shifting a variable acts on its CF by a phase and an argument rescaling.

\varphi_{aX+b}(t)=e^{ibt}\,\varphi_X(at) \qquad (6)

where: $a,b$ — real constants (a scaling and a shift). This is the tool that reduces any Gaussian to the standard one and supplies the $1/(\sigma\sqrt{n})$ rescaling in the CLT. Because the CF is a Fourier transform, it is invertible: the distribution is recovered uniquely.

f_X(x)=\frac{1}{2\pi}\int_{-\infty}^{\infty} e^{-itx}\,\varphi_X(t)\,\mathrm{d}t \qquad (7)

where: $f_X(x)$ — recovered density, valid when $\varphi_X\in L^1$ ; a more general inversion formula recovers the cumulative distribution for any law. Reading: two distributions with the same CF are identical — this is the uniqueness theorem that licenses "recognize the transform, name the law". The standard laws have compact, memorable transforms — the Gaussian above all.

\varphi_X(t)=\exp\!\left(i\mu t-\tfrac{1}{2}\sigma^2 t^2\right),\qquad \varphi_X(t)=e^{\lambda(e^{it}-1)}\ (\text{Poisson}),\qquad \varphi_X(t)=(1-p+pe^{it})^n\ (\text{binomial}) \qquad (8)

where: $\mu,\sigma^2$ — Gaussian mean and variance; $\lambda$ — Poisson rate (equal to both mean and variance); $n,p$ — binomial trial count and success probability. Reading: each closed form is a lookup key for inversion — and the Gaussian's CF is again a Gaussian, the fixed-point structure behind the CLT.

Derivation AWhy differentiating the transform generates the moments

The goal is to justify the name "moment-generating function" rigorously: to show that the single analytic function $M_X(t)$ carries the entire sequence of raw moments $\mathbb{E}[X^n]$ as the coefficients of its Taylor series, so that repeated differentiation at the origin reads them off one by one. We expand the exponential weight, exchange the sum with the expectation under the convergence hypothesis, and identify the Taylor coefficients — then note where the same argument survives for the characteristic function.

Step 1. Expand the weight as a power series in $t$ : since $e^{tX}=\sum_{n=0}^\infty (tX)^n/n!$ converges for every value of the random variable, we may write $e^{tX}=\sum_n t^nX^n/n!$ as an identity between random variables. Why: the moments are hidden inside the powers $X^n$ , and the exponential is the generating device that exposes all of them at once.

Step 2. Take expectations term by term: $M_X(t)=\mathbb{E}\big[\sum_n t^nX^n/n!\big]=\sum_n \mathbb{E}[X^n]\,t^n/n!$ . The interchange of the infinite sum and the expectation is legitimate precisely when $M_X$ exists in a neighbourhood of the origin, i.e. when $\mathbb{E}[e^{t_0|X|}]<\infty$ for some $t_0>0$ , by dominated convergence. Why: this is the exact point where "the MGF exists" does real work — for a Cauchy or log-normal law the interchange fails and the series is meaningless.

Step 3. Match Taylor coefficients. The coefficient of $t^n$ in $M_X(t)$ is $\mathbb{E}[X^n]/n!$ , so differentiating $n$ times and setting $t=0$ kills every other term and leaves $M_X^{(n)}(0)=\mathbb{E}[X^n]$ . For the characteristic function the identical expansion of $e^{itX}$ gives $\varphi_X^{(n)}(0)=i^n\mathbb{E}[X^n]$ , hence $\mathbb{E}[X^n]=(-i)^n\varphi_X^{(n)}(0)$ . Why: this converts an analytic operation, differentiation, into a probabilistic one, moment extraction — and it works for the CF whenever the moment exists, without needing the whole series to converge. *Verified: for the Exponential-rate-2 law, $M_X(t)=2/(2-t)$ gives $M'(0)=M''(0)=1/2$ , so mean $0.5$ , $\mathbb{E}[X^2]=0.5$ , variance $0.25$ — matching $1/\lambda$ and $1/\lambda^2$ (Worked Example 1).*

Go deeper · Derivation B · 1 minThe characteristic function of the Gaussian is again a Gaussian

Derivation BThe characteristic function of the Gaussian is again a Gaussian

The goal is to compute $\varphi_X(t)$ for $X\sim\mathcal{N}(\mu,\sigma^2)$ from the definition and to establish the self-reproducing form $\exp(i\mu t-\tfrac12\sigma^2 t^2)$ — the single most consequential transform in probability, the fixed point that every finite-variance sum flows toward. We standardize with the affine rule, complete the square in the complex exponent, shift the contour of a Gaussian integral, reassemble, then confirm the first two moments.

Step 1. Standardize. Write $X=\mu+\sigma Z$ with $Z\sim\mathcal{N}(0,1)$ ; the affine rule (6) gives $\varphi_X(t)=e^{i\mu t}\,\varphi_Z(\sigma t)$ , reducing the problem to the standard normal. Why: the shift contributes only a phase and the scale only rescales the argument, so all the work is a single standard integral.

Step 2. Write the standard-normal integral with $s=\sigma t$ : $\varphi_Z(s)=\frac{1}{\sqrt{2\pi}}\int_{-\infty}^\infty e^{isz}e^{-z^2/2}\,\mathrm{d}z$ . Why: this is the Fourier transform of the Gaussian kernel, the one integral that reproduces itself.

Step 3. Complete the square in the exponent: $isz-\tfrac12 z^2=-\tfrac12(z-is)^2-\tfrac12 s^2$ . Why: pulling out the $z$ -independent factor $e^{-s^2/2}$ leaves a pure Gaussian in the shifted variable $z-is$ .

Step 4. Shift the contour and integrate: $\frac{1}{\sqrt{2\pi}}\int e^{-(z-is)^2/2}\,\mathrm{d}z=1$ , the shift from the real axis to the line $\mathrm{Im}\,z=-s$ being justified because $e^{-z^2/2}$ is entire and decays in the strip. Hence $\varphi_Z(s)=e^{-s^2/2}$ . Why: the normalization of the Gaussian is exactly what makes the transform reproduce the same functional form.

Step 5. Collect factors: substitute $s=\sigma t$ and multiply by the phase, $\varphi_X(t)=e^{i\mu t}\,e^{-\sigma^2 t^2/2}=\exp(i\mu t-\tfrac12\sigma^2 t^2)$ . Why: this closed form is the lookup key for every Gaussian sum and the target of the CLT limit. *Verified: $\varphi_X'(0)=i\mu$ gives mean $\mu$ ; $\varphi_X''(0)=-(\sigma^2+\mu^2)$ gives $\mathbb{E}[X^2]=\sigma^2+\mu^2$ , hence variance $\sigma^2$ . Since $\ln\varphi_X$ is exactly quadratic, $\kappa_1=\mu$ , $\kappa_2=\sigma^2$ , and all higher cumulants vanish.*

Go deeper · Derivation C · 1 minThe Central Limit Theorem from a Taylor expansion of the characteristic function

Derivation CThe Central Limit Theorem from a Taylor expansion of the characteristic function

The goal is to prove that the standardized sum of independent, identically distributed variables with finite variance converges in distribution to a standard Gaussian, using nothing but the product rule and a two-term Taylor expansion of the characteristic function near the origin — the cleanest known route, which sidesteps convolving $n$ densities. We expand $\varphi_X$ to second order, raise it to the $n$ -th power under the standardizing rescaling, take the limit, and invoke Lévy's continuity theorem to convert CF convergence into convergence in distribution.

Step 1. Localize. Assume mean $0$ and variance $\sigma^2<\infty$ ; then $\varphi_X(s)=1-\tfrac12\sigma^2 s^2+o(s^2)$ as $s\to0$ , the linear term absent because the mean vanishes. Why: only two moments are needed — this is why the CLT holds for laws with no moments beyond the second, and why the variance is the sole scale that enters.

Step 2. Factorize the standardized sum. With $S_n=\sum_{k=1}^n X_k$ and $Z_n=S_n/(\sigma\sqrt{n})$ , the product rule (5) plus the affine rule (6) give $\varphi_{Z_n}(t)=\big[\varphi_X\!\big(t/(\sigma\sqrt n)\big)\big]^n$ . Why: independence has turned an $n$ -fold convolution into an $n$ -th power, and the $1/(\sigma\sqrt n)$ is exactly the scaling that keeps the variance fixed at one.

Step 3. Insert the expansion. Substituting $s=t/(\sigma\sqrt n)$ into Step 1, the $\sigma^2$ cancels and $\varphi_{Z_n}(t)=\big[1-\tfrac{t^2}{2n}+o(1/n)\big]^n$ . Why: the rescaling has tuned the quadratic term to size $1/n$ , precisely the regime where $n$ -fold powers converge to an exponential.

Step 4. Take the limit using $(1+a/n)^n\to e^a$ : $\lim_{n\to\infty}\big(1-\tfrac{t^2}{2n}\big)^n=e^{-t^2/2}$ . Why: the higher-order $o(1/n)$ terms vanish in the limit, leaving exactly the standard-normal CF derived above.

Step 5. Convert to distribution. By Lévy's continuity theorem, pointwise convergence of characteristic functions to a limit that is continuous at $t=0$ implies convergence in distribution, so $Z_n\xrightarrow{d}\mathcal{N}(0,1)$ . Why: the continuity theorem is the bridge that makes the CF a proof tool and not merely a bookkeeping device. *Verified: the limit $e^{-t^2/2}$ is identically the standard-normal CF of the previous derivation, confirming Gaussian universality; equivalently $\ln\varphi_{Z_n}\to-t^2/2$ , so every cumulant of order $\ge3$ is suppressed by a power of $n$ .*

Limiting cases

The behaviour of $\varphi_X(t)$ encodes the regularity and tail structure of $X$ at a glance:

Case	Result	Meaning
Origin, moments exist	$\varphi_X(t)=1+i\mu t-\tfrac{1}{2}(\sigma^2+\mu^2)t^2+o(t^2)$	if $\mathbb{E}[X^n]$ exists then $\varphi$ is $n$ -times differentiable at $0$ , and its Taylor coefficients are the moments
Heavy tail (Cauchy)	$\varphi_X(t)=e^{-\lvert t\rvert}$ , while $M_X(t)=+\infty$ for $t\neq0$	CF exists but is non-differentiable at $0$ (no finite mean); the MGF does not exist at all
Smooth density, large $\lvert t\rvert$	$\varphi_X(t)\to0$ as $\lvert t\rvert\to\infty$	Riemann–Lebesgue: an integrable density has a decaying CF, and faster decay means a smoother density
Lattice variable	$\lvert\varphi_X(t)\rvert=1$ for some $t\neq0$	periodicity of $\lvert\varphi\rvert$ signals a discrete law supported on a lattice $a+h\mathbb{Z}$

Common trapThe moment-generating function always exists and its moments always determine the law

Neither half is safe. The MGF requires $e^{tX}$ to be integrable near $t=0$ ; for the Cauchy, log-normal, or any power-law tail this fails and $M_X(t)=\infty$ for every $t\neq0$ , so the "generating function" generates nothing. Worse, even when all moments are finite they need not pin down the distribution: the log-normal shares its entire moment sequence with a whole family of other laws, so "matching moments" can silently identify the wrong one. The characteristic function has neither defect — it exists for every distribution and, by Fourier inversion, determines the law uniquely. When in doubt, work with $\varphi_X(t)=\mathbb{E}[e^{itX}]$ , not the MGF.

Worked examples

1 · Reading moments off an Exponential MGF (confidence builder)

Let $X$ be Exponential with rate $\lambda=2$ , density $f_X(x)=2e^{-2x}$ for $x\ge0$ . Find its moment-generating function, then extract the mean and variance by differentiation, and confirm against the closed forms.

Step 1. Assemble the MGF from the definition: $M_X(t)=\int_0^\infty e^{tx}\,2e^{-2x}\,\mathrm{d}x=2\int_0^\infty e^{-(2-t)x}\,\mathrm{d}x=\dfrac{2}{2-t}$ , valid for $t<2$ . The restriction $t<2$ is the neighbourhood-of-origin convergence in action — push $t$ up to the rate and the integral diverges. $M_X(t)=2/(2-t)$ , defined for $t<2$ .

Step 2. First derivative for the mean: $M_X'(t)=\dfrac{2}{(2-t)^2}$ , so $M_X'(0)=2/4=\mathbf{0.5}$ . This is $\mathbb{E}[X]=1/\lambda$ , exactly.

Step 3. Second derivative for the second moment: $M_X''(t)=\dfrac{4}{(2-t)^3}$ , so $M_X''(0)=4/8=\mathbf{0.5}=\mathbb{E}[X^2]$ . Then the variance is $\mathrm{Var}(X)=\mathbb{E}[X^2]-(\mathbb{E}[X])^2=0.5-0.25=\mathbf{0.25}=1/\lambda^2$ . Both match the standard Exponential results $1/\lambda$ and $1/\lambda^2$ . Mean 0.5, variance 0.25 — read off in two derivatives.

2 · A Central Limit estimate for a sum of uniforms

A classic random-number recipe adds twelve independent $\mathrm{Uniform}(0,1)$ variables and subtracts $6$ to manufacture an approximate standard normal. Justify it with characteristic functions and estimate $\mathbb{P}(S>7)$ , where $S=\sum_{k=1}^{12}U_k$ .

Step 1. Fix the moments. A single $\mathrm{Uniform}(0,1)$ has mean $\tfrac12$ and variance $\tfrac{1}{12}$ . By additivity of cumulants (equivalently, the product rule for CFs), the sum $S$ of twelve independent copies has mean $12\cdot\tfrac12=\mathbf{6}$ and variance $12\cdot\tfrac{1}{12}=\mathbf{1}$ . Subtracting $6$ centres it; the variance is already $1$ , which is why this recipe needs no rescaling.

Step 2. Justify the Gaussian shape. Each uniform has CF $\varphi_U(t)=(e^{it}-1)/(it)$ , so the centred sum has CF $\big[e^{-it/2}\varphi_U(t)\big]^{12}=\big[\mathrm{sinc}(t/2)\big]^{12}$ with $\mathrm{sinc}(x)=\sin x/x$ . Near the origin $\mathrm{sinc}(t/2)=1-t^2/24+o(t^2)$ , so the twelfth power is $\big[1-t^2/24+\cdots\big]^{12}\to e^{-t^2/2}$ — the standard-normal CF. Twelve is already large enough that $S-6\approx\mathcal{N}(0,1)$ over the central range.

Step 3. Estimate the probability. $\mathbb{P}(S>7)=\mathbb{P}(S-6>1)\approx\mathbb{P}(Z>1)$ with $Z\sim\mathcal{N}(0,1)$ . From the standard normal tail, $\mathbb{P}(Z>1)=\mathbf{0.159}$ — roughly one draw in six exceeds $7$ , obtained without convolving twelve densities, purely by recognizing the limiting CF. $S-6\approx\mathcal{N}(0,1)$ , and $\mathbb{P}(S>7)\approx0.159$ .

3 · Adding two independent Poissons by multiplying transforms

Let $X\sim\mathrm{Poisson}(2)$ and $Y\sim\mathrm{Poisson}(3)$ be independent. Identify the law of $X+Y$ and compute $\mathbb{P}(X+Y=4)$ .

Step 1. Multiply the characteristic functions. Each Poisson has $\varphi_X(t)=e^{\lambda(e^{it}-1)}$ , so by the product rule $\varphi_{X+Y}(t)=e^{2(e^{it}-1)}\,e^{3(e^{it}-1)}=e^{5(e^{it}-1)}$ . This is the CF of $\mathrm{Poisson}(5)$ , so $X+Y\sim\mathrm{Poisson}(5)$ — closure in one line by uniqueness of the CF.

Step 2. Evaluate the pmf. $\mathbb{P}(X+Y=4)=e^{-5}\dfrac{5^4}{4!}=e^{-5}\dfrac{625}{24}$ . With $e^{-5}=0.006738$ and $625/24=26.04$ , the product is $\mathbf{0.175}$ . $X+Y\sim\mathrm{Poisson}(5)$ and $\mathbb{P}(X+Y=4)\approx0.175$ .

Physics of the portrait

The characteristic function is where two centuries of probability meet the Fourier analysis of physics: a device Laplace used to sum errors, that Cauchy used to exhibit a monster with no mean, and that Lévy finally forged into the exact tool for the limit theorems.

Origins · 1 minPaul Lévy and how this was discovered

Paul Lévy

1886–1971 · the mathematician who forged the modern theory of characteristic functions

Paul Lévy was the figure who turned the characteristic function from a computational trick into a complete theory. Trained as an engineer at the École Polytechnique — where he would later teach analysis for decades — he came to probability comparatively late, and reshaped it. His 1925 monograph Calcul des probabilités set out the characteristic function as the central object of the subject and proved the result that bears his name: the continuity theorem, which states that if a sequence of characteristic functions converges pointwise to a function continuous at the origin, then the corresponding distributions converge, and the limit is itself a characteristic function. This is the theorem that makes the CF a genuine instrument of proof — without it the limit $(1-t^2/2n)^n\to e^{-t^2/2}$ would be only suggestive. Lévy went on to characterize the full class of limit laws for sums of independent variables (the stable and infinitely divisible distributions, several of which now carry his name), to found the modern theory of Brownian motion, and to introduce the martingale ideas his student Doob would systematize — much of the language physicists now use for diffusion and noise. The Fourier transform of a probability law, certain to determine it, is his lasting monument.

History

1812

— Laplace deploys the Fourier-type transform of a distribution in his Théorie analytique des probabilités

1853

— Cauchy exhibits the law with characteristic function e^−|t| that has no mean

1901

— Lyapunov proves the central limit theorem rigorously via characteristic functions

1922

— Lindeberg gives the definitive finite-variance condition for the CLT

1925

— Lévy's Calcul des probabilités founds the modern theory and states the continuity theorem

Going further

Texts. Feller, An Introduction to Probability Theory and Its Applications, Vol. 2, Chs. XV–XVI (characteristic functions and the CLT) · Grimmett & Stirzaker, Probability and Random Processes, Ch. 5 · Stone & Goldbart, Mathematics for Physics, chapter on probability and Fourier methods · Arfken, Weber & Harris, Mathematical Methods for Physicists, probability chapter · Riley, Hobson & Bence, Mathematical Methods for Physics and Engineering, statistics chapter.

Provenance. The definitions and the moment/cumulant relations follow Feller Vol. 2 and Stone & Goldbart; the Gaussian CF $\exp(i\mu t-\tfrac12\sigma^2 t^2)$ , the Poisson CF $e^{\lambda(e^{it}-1)}$ , the binomial CF $(1-p+pe^{it})^n$ , and the Cauchy CF $e^{-|t|}$ are standard textbook values (Grimmett & Stirzaker; Feller Vol. 2). A verifier should: (1) re-derive the Gaussian CF by completing the square and confirm $\varphi'(0)=i\mu$ , $\varphi''(0)=-(\sigma^2+\mu^2)$ ; (2) check the Exponential-rate-2 MGF $2/(2-t)$ gives mean $0.5$ , variance $0.25$ ; (3) confirm the CLT limit $(1-t^2/2n)^n\to e^{-t^2/2}$ and the uniform-sum estimate $\mathbb{P}(S>7)\approx0.159$ ; (4) verify Poisson additivity by multiplying CFs and $\mathbb{P}(\mathrm{Poisson}(5)=4)\approx0.175$ . The inversion and Lévy continuity theorems are as in Feller Vol. 2.

Prerequisites. Expectation, Variance and Moments · Random Variables and Probability Distributions · Cumulative Distribution Functions and Change of Variables · Axioms of Probability. Leads to. Central Limit Theorem · Laws of Large Numbers and Convergence of Random Variables · Gaussian (Normal) Distribution · Poisson Distribution.

Ask the tutor: Ask me to invert a characteristic function you are stuck on, or to show step by step how the cumulants of your favourite distribution collapse under the Central Limit Theorem.

When should I use the characteristic function instead of the moment-generating function?

Use the characteristic function whenever existence or uniqueness could be in doubt — which, in practice, is most of the time. The MGF is often handier for hand computation because it avoids the imaginary unit, and for light-tailed laws (Gaussian, Poisson, exponential, bounded variables) it is perfectly reliable. But the moment you touch a heavy-tailed law — Cauchy, log-normal, any power-law tail — the MGF diverges and is useless, while the CF sails through because its integrand has modulus one. Since the CF also guarantees uniqueness of inversion, probabilists adopt it as the default and treat the MGF as a shortcut valid only when it exists near the origin.

Do moments always determine a distribution uniquely?

No, and this is a genuine trap. Having a full sequence of finite moments is not enough to pin down a law. The classic counterexample is the log-normal distribution: an entire family of distinct laws share its every moment, so the moment problem is "indeterminate" there. A sufficient condition for determinacy is that the MGF exists in a neighbourhood of the origin (Carleman's condition is weaker but similar in spirit) — which is why light-tailed laws are safe. The characteristic function sidesteps the whole issue: two laws with the same CF are always identical, because the CF is an invertible Fourier transform. If you need certainty, match characteristic functions, not moments.

What is the difference between a moment and a cumulant, and why prefer cumulants?

Moments $\mathbb{E}[X^n]$ are the raw averages of powers; cumulants $\kappa_n$ are their "connected" repackaging, the Taylor coefficients of $\ln M_X(t)$ rather than of $M_X(t)$ . The first two agree with the familiar quantities — $\kappa_1$ is the mean, $\kappa_2$ the variance — and the third is the central third moment, but from the fourth on they differ. The decisive advantage is additivity: for independent variables cumulants simply add, $\kappa_n(X+Y)=\kappa_n(X)+\kappa_n(Y)$ , whereas moments mix through cross-terms. This makes cumulants the natural language for sums, for the CLT, and for perturbative expansions in physics.

Why does the Central Limit Theorem single out the Gaussian and nothing else?

Because the Gaussian is the fixed point of the operation "add independent copies and rescale". In cumulant language the reason is transparent: standardizing a sum of $n$ variables divides the $n$ -th cumulant by $n^{n/2-1}$ , so every cumulant beyond the second is driven to zero as $n\to\infty$ , leaving only the mean (set to $0$ ) and the variance (set to $1$ ). A law whose cumulants vanish beyond second order is exactly a Gaussian — and Marcinkiewicz's theorem confirms no other has a polynomial cumulant-generating function. At the level of the characteristic function this is the statement that $[\varphi_X(t/\sqrt n)]^n\to e^{-t^2/2}$ for every finite-variance law, independent of the details of $\varphi_X$ beyond its curvature at the origin.

The characteristic function is complex-valued — how do I read information off it?

Its value at the origin is always $1$ (total probability) and its modulus never exceeds $1$ . The behaviour near $t=0$ carries the moments: the slope $\varphi'(0)=i\mu$ encodes the mean, the curvature $\varphi''(0)=-(\sigma^2+\mu^2)$ the second moment, and so on through equation (3). Symmetry about zero makes $\varphi$ real and even; a shift multiplies $\varphi$ by a pure phase $e^{ibt}$ . The decay of $\varphi$ at large $|t|$ reports the smoothness of the density (Riemann–Lebesgue), while any persistence of $|\varphi|=1$ at nonzero $t$ signals a lattice-supported discrete law. To recover the density itself you invert — equation (7).

Is the characteristic function really the same object as the Fourier transform used in physics?

Yes — up to a sign and normalization convention. The characteristic function $\varphi_X(t)=\int e^{itx}f_X(x)\,\mathrm{d}x$ is precisely the Fourier transform of the probability density, with the probabilist's convention placing the $2\pi$ in the inversion integral (7). Every theorem you know from physics carries over: the convolution theorem becomes the product rule for sums; the uncertainty principle becomes the trade-off between a concentrated density and a slowly decaying CF. This is not an analogy but an identity, and it is why the entire apparatus of Fourier analysis — fast transforms, saddle points, causality relations — is directly available to probability and statistical physics.