Characteristic Functions and Moment Generating Functions

Characteristic functions and moment generating functions are integral transforms of a probability distribution that encode all its information in a single function. The characteristic function

φ_{X} (t) = E [e^{i tX}]

always exists; the moment generating function

M_{X} (t) = E [e^{tX}]

may not. They linearize convolution (sums of independent variables become products), generate moments by differentiation, and uniquely determine the distribution. Used everywhere in probability theory, statistics, and the proof of the Central Limit Theorem; developed by Lyapunov, Lévy, and others in the late 19th–early 20th century.

Given a random variable $X$ , its moment generating function (MGF) is $M_{X} (t) = E [e^{tX}]$ and its characteristic function (CF) is $φ_{X} (t) = E [e^{i tX}]$ , where $i = - 1$ . These functions transform the distribution into a domain where many operations become algebraic: the distribution of a sum of independent variables is recovered by multiplying their transforms, and the $n$ -th moment is recovered by differentiating $n$ times at the origin. The CF is the more powerful of the two because $∣ e^{i tX} ∣ = 1$ guarantees its existence for every distribution, while the MGF requires exponential tail decay and can blow up. A surprising fact: the characteristic function is essentially the Fourier transform of the probability density, which means the entire machinery of Fourier analysis — inversion, Plancherel, convolution theorems — applies directly to probability.

The Core Insight

A probability distribution is fully captured not only by its density or CDF but by any one of several transforms. By replacing the variable $X$ with $e^{i tX}$ or $e^{tX}$ and taking expectations, we convert convolution into multiplication and moments into derivatives. This is exactly why these transforms are the natural tool for studying sums of independent random variables — the engine behind limit theorems.

Formal Statement

Moment Generating Function

For a random variable $X$ , the moment generating function is

M_{X} (t) = E [e^{tX}] = ⎩ ⎨ ⎧ x \sum e^{t x} p_{X} (x) \int_{- \infty}^{\infty} e^{t x} f_{X} (x) d x (discrete) (continuous)

⟳Apply the Law of the Unconscious Statistician (LOTUS) to the function

g (x) = e^{t x}

, summing or integrating against the distribution.

Here $t \in R$ , $p_{X}$ is the probability mass function, and $f_{X}$ is the probability density function. The MGF is defined only for those $t$ where the sum/integral converges; we require convergence on an open interval containing $0$ .

Characteristic Function

For any random variable $X$ , the characteristic function is

φ_{X} (t) = E [e^{i tX}] = E [cos tX] + i E [sin tX], t \in R .

⟳Use Euler's formula

e^{i θ} = cos θ + i sin θ

and split the expectation into real and imaginary parts.

Here $i^{2} = - 1$ . Because $∣ e^{i tX} ∣ = 1$ , the integrand is bounded, so $φ_{X} (t)$ exists and is finite for every $t$ and every distribution.

Moments from Derivatives

If $M_{X}$ exists in a neighborhood of $0$ , then $X$ has finite moments of all orders and

E [X^{n}] = M_{X}^{(n)} (0) = \frac{d ^{n}}{d t ^{n}} M_{X} (t)_{t = 0} .

⟳Differentiate

e^{tX}

under the expectation

n

times to bring down a factor

X^{n}

, then set

t = 0

e^{tX} \to 1

Equivalently, the Taylor series $M_{X} (t) = \sum_{n = 0}^{\infty} \frac{E [ X ^{n} ]}{n !} t^{n}$ holds. For the characteristic function, when $E [∣ X ∣^{n}] < \infty$ ,

E [X^{n}] = \frac{1}{i ^{n}} φ_{X}^{(n)} (0) = (- i)^{n} φ_{X}^{(n)} (0) .

⟳Differentiate

e^{i tX}

to pull down

i X

each time; after

n

derivatives the prefactor is

i^{n}

, so divide by it.

Uniqueness and Convolution

Uniqueness. The characteristic function determines the distribution uniquely: if

φ_{X} = φ_{Y}

then

X

and

Y

have the same distribution. The same holds for the MGF when it exists on an open interval around

0

. Convolution / Independence. If

X

and

Y

are independent, then for the sum

S = X + Y

φ_{X + Y} (t) = φ_{X} (t) φ_{Y} (t), M_{X + Y} (t) = M_{X} (t) M_{Y} (t) .

⟳

E [e^{i t (X + Y)}] = E [e^{i tX} e^{i t Y}]

; independence factors the expectation of a product into a product of expectations.

Linear transformation. For constants

a, b

φ_{a X + b} (t) = e^{i t b} φ_{X} (a t), M_{a X + b} (t) = e^{t b} M_{X} (a t) .

⟳

E [e^{i t (a X + b)}] = e^{i t b} E [e^{i (a t) X}]

; pull the constant exponent out and rescale the argument.

Why It Holds / Derivation

Moments by Differentiation

We justify equation [3], assuming $M_{X} (t)$ converges in $(- δ, δ)$ for some $δ > 0$ .

Step 1. Write the definition

M_{X} (t) = E [e^{tX}]

. Step 2. Expand the exponential in its power series:

e^{tX} = \sum_{n = 0}^{\infty} \frac{( tX ) ^{n}}{n !}

. Step 3. Convergence of

M_{X}

on an open interval permits interchanging expectation and summation (dominated convergence / Fubini):

M_{X} (t) = n = 0 \sum \infty \frac{E [ X ^{n} ]}{n !} t^{n} .

Step 4. This is a power series in

t

. A power series equals the Taylor series of the function it represents, so the coefficient of

t^{n}

M_{X}^{(n)} (0) / n!

. Step 5. Matching coefficients:

\frac{E [ X ^{n} ]}{n !} = \frac{M _{X}^{(n)} ( 0 )}{n !}

, hence

E [X^{n}] = M_{X}^{(n)} (0)

■

Convolution Theorem for Sums

We justify equation [5] for independent $X, Y$ .

Step 1. By definition,

φ_{X + Y} (t) = E [e^{i t (X + Y)}]

. Step 2. Algebra of exponents:

e^{i t (X + Y)} = e^{i tX} \cdot e^{i t Y}

. Step 3. Since

X

and

Y

are independent, the functions

e^{i tX}

and

e^{i t Y}

are independent, so the expectation of the product factors:

E [e^{i tX} e^{i t Y}] = E [e^{i tX}] E [e^{i t Y}] .

Step 4. Recognize each factor:

= φ_{X} (t) φ_{Y} (t)

. The identical argument with

t

in place of

i t

proves the MGF version.

■

Paul Lévy | 1886–1971

Paul Lévy was the central architect of the modern theory of characteristic functions. Building on Lyapunov's use of them in proving the Central Limit Theorem, Lévy established the continuity theorem (pointwise convergence of CFs corresponds to convergence in distribution), developed the inversion formula in full generality, and classified the infinitely divisible and stable distributions through their characteristic functions. His 1925 Calcul des probabilités made the characteristic function the standard tool of rigorous probability. Lévy also pioneered stochastic processes (Lévy processes, Brownian motion), and his name is attached to a vast range of probabilistic objects.

Building Intuition

Think of a probability distribution as a "signal" — a shape spread over the real line. The characteristic function is its Fourier transform: it re-expresses the same information in terms of frequencies $t$ rather than positions $x$ . Two facts then become intuitive.

First, why sums become products. Adding independent random variables means convolving their densities (smearing one shape over the other). The Fourier transform turns convolution into ordinary multiplication — so in the transform domain, addition of variables is just multiplication of functions. This is the same principle that makes Fourier analysis so useful in signal processing.

Second, why derivatives give moments. Near $t = 0$ , $e^{i tX} \approx 1 + i tX - \frac{1}{2} t^{2} X^{2} + \dots$ . Taking expectations, the coefficients are precisely $1, i E [X], - \frac{1}{2} E [X^{2}], \dots$ The local behavior of the transform at the origin encodes the moments — small $t$ probes large-scale features (mean, variance) of the distribution.

The MGF does the same thing but along the real axis, which is why it can diverge: $e^{tX}$ grows without bound, so heavy-tailed distributions overwhelm it. The CF's $e^{i tX}$ merely rotates on the unit circle, never growing, so it is always tame.

The two transforms: MGF integrates against a growing exponential (may diverge), CF against a bounded oscillation (always exists)

<svg viewBox="0 0 700 380" xmlns="http://www.w3.org/2000/svg">
<rect width="700" height="380" fill="#0a0e17"/>
<text x="350" y="28" fill="#e2e8f0" font-size="17" text-anchor="middle" font-family="sans-serif">Weighting kernels acting on a distribution</text>

<!-- axis -->
<line x1="60" y1="200" x2="640" y2="200" stroke="#94a3b8" stroke-width="1.5"/>
<line x1="350" y1="60" x2="350" y2="340" stroke="#94a3b8" stroke-width="1"/>
<text x="648" y="205" fill="#94a3b8" font-size="13" font-family="sans-serif">x</text>

<!-- density bump (key curve) -->
<path d="M 130 200 Q 230 70 350 70 Q 470 70 570 200" fill="none" stroke="#d4a853" stroke-width="2.5"/>
<text x="350" y="60" fill="#d4a853" font-size="13" text-anchor="middle" font-family="sans-serif">f(x): distribution</text>

<!-- MGF growing exponential -->
<path d="M 200 200 C 320 198 430 150 520 80" fill="none" stroke="#7da4d4" stroke-width="2" stroke-dasharray="5 4"/>
<text x="525" y="78" fill="#7da4d4" font-size="12" font-family="sans-serif">e^{tx} (grows → may diverge)</text>

<!-- CF oscillation (bounded) -->
<path d="M 80 260 Q 110 220 140 260 T 200 260 T 260 260 T 320 260 T 380 260 T 440 260 T 500 260 T 560 260 T 620 260" fill="none" stroke="#84cc84" stroke-width="2"/>
<text x="80" y="300" fill="#84cc84" font-size="12" font-family="sans-serif">cos(tx), sin(tx): bounded by 1 (always converges)</text>

<line x1="80" y1="240" x2="620" y2="240" stroke="#84cc84" stroke-width="0.5" stroke-dasharray="2 3"/>
<line x1="80" y1="280" x2="620" y2="280" stroke="#84cc84" stroke-width="0.5" stroke-dasharray="2 3"/>
<text x="630" y="244" fill="#84cc84" font-size="11" font-family="sans-serif">+1</text>
<text x="630" y="284" fill="#84cc84" font-size="11" font-family="sans-serif">-1</text>
</svg>

Worked Examples

Exponential Distribution: MGF and Moments

Given:

X \sim Exp (λ)

with density

f_{X} (x) = λ e^{- λ x}

for

x \geq 0

λ = 2

. Step 1 — Set up the MGF.

M_{X} (t) = \int_{0}^{\infty} e^{t x} λ e^{- λ x} d x = λ \int_{0}^{\infty} e^{- (λ - t) x} d x .

Step 2 — Evaluate (converges only for $t < λ$ ).

M_{X} (t) = λ \cdot \frac{1}{λ - t} = \frac{λ}{λ - t}, t < λ .

Step 3 — First derivative for the mean.

M_{X}^{'} (t) = \frac{λ}{( λ - t ) ^{2}}, E [X] = M_{X}^{'} (0) = \frac{λ}{λ ^{2}} = \frac{1}{λ} .

Step 4 — Second derivative for $E [X^{2}]$ .

M_{X}^{''} (t) = \frac{2 λ}{( λ - t ) ^{3}}, E [X^{2}] = M_{X}^{''} (0) = \frac{2 λ}{λ ^{3}} = \frac{2}{λ ^{2}} .

Step 5 — Variance and numbers with

λ = 2

E [X] = \frac{1}{2} = 0.5, E [X^{2}] = \frac{2}{4} = 0.5, Var (X) = 0.5 - 0.25 = 0.25.

M_{X} (t) = \frac{2}{2 - t} (t < 2), E [X] = 0.5, Var (X) = 0.25

Standard Normal: Characteristic Function and Sum of Normals

Given:

Z \sim N (0, 1)

with density

f (z) = \frac{1}{2 π} e^{- z^{2} /2}

. Step 1 — Set up the CF.

φ_{Z} (t) = \int_{- \infty}^{\infty} e^{i t z} \frac{1}{2 π} e^{- z^{2} /2} d z .

Step 2 — Complete the square in the exponent.

i t z - \frac{z ^{2}}{2} = - \frac{1}{2} (z^{2} - 2 i t z) = - \frac{1}{2} (z - i t)^{2} - \frac{t ^{2}}{2} .

Step 3 — Integrate (the shifted Gaussian integrates to

2 π

φ_{Z} (t) = e^{- t^{2} /2} \cdot \frac{1}{2 π} \int_{- \infty}^{\infty} e^{- (z - i t)^{2} /2} d z = e^{- t^{2} /2} .

Step 4 — Apply to a sum. Let

X_{1}, \dots, X_{n}

be i.i.d.

N (0, 1)

and

S = \sum X_{k}

. By the convolution property [5],

φ_{S} (t) = k = 1 \prod n e^{- t^{2} /2} = e^{- n t^{2} /2} .

Step 5 — Recognize the result.

e^{- n t^{2} /2}

is the CF of

N (0, n)

(a normal with variance

n

). For

n = 4

: variance

= 4

, standard deviation

= 2

φ_{Z} (t) = e^{- t^{2} /2}, S = k = 1 \sum 4 Z_{k} \sim N (0, 4), σ_{S} = 2

Common Misconceptions

The MGF need not exist. Many students assume every distribution has an MGF. It does not. The MGF requires

E [e^{tX}] < \infty

on an open interval around

0

, which fails for heavy-tailed distributions. The Cauchy distribution, the log-normal, and any distribution with infinitely many infinite-or-undefined moments have no MGF (or only the degenerate value at

t = 0

). The characteristic function, by contrast, always exists — this is its primary advantage and the reason rigorous probability theory prefers it.

A second pitfall: equation [3] gives moments only if the MGF is finite in a neighborhood of $0$ . Finiteness at a single point $t = 0$ (where $M_{X} (0) = 1$ trivially) tells you nothing.

Distributions with Equal Moments but Different Laws

A common belief is that "knowing all the moments determines the distribution." This is false without an analyticity condition. The classic counterexample uses the log-normal distribution. Let $X$ be log-normal. There exist densities of the form

f_{a} (x) = f_{X} (x) (1 + a sin (2 π ln x)), ∣ a ∣ \leq 1,

which are genuine probability densities (different for different

a

) yet share exactly the same moments of every order as the log-normal. Since they have identical moment sequences but different distributions, the moment sequence does not determine the law. The reason: the log-normal's MGF does not exist (its tails are too heavy), so its moments grow too fast for the Carleman condition

\sum_{n} (E [X^{2 n}])^{- 1/2 n} = \infty

to hold. The characteristic functions of these distributions do differ — and the CF, unlike the moment sequence, always determines the law uniquely.

Connections

The characteristic function is, up to sign conventions, the Fourier transform of the density: $φ_{X} (t) = \hat{f}_{X} (- t)$ . This ties probability directly to harmonic analysis — the inversion formula, Parseval/Plancherel identities, and convolution theorems all carry over. See Fourier Transform.

The MGF is the two-sided Laplace transform of the density evaluated along the real axis, linking it to Laplace Transform methods used in differential equations and queueing theory. The cumulant generating function $K_{X} (t) = lo g M_{X} (t)$ generates cumulants and is additive for independent sums, connecting to Cumulants.

The whole apparatus is the proof engine behind the Central Limit Theorem: standardizing a sum and showing its CF converges to $e^{- t^{2} /2}$ , then invoking Lévy's continuity theorem and Convergence in Distribution. The closely related probability generating function $G_{X} (z) = E [z^{X}]$ serves integer-valued variables; see Probability Generating Function.

Real-World Applications

Central Limit Theorem proofs. The cleanest proof expands the CF of a standardized sum and shows it converges to $e^{- t^{2} /2}$ . This underpins virtually all of inferential statistics.

Insurance and risk (aggregate claims). The total claim $S = \sum X_{i}$ over a random number $N$ of claims has CF computed by compounding; actuaries use Panjer recursion and transform inversion to find ruin probabilities.

Queueing theory. The waiting-time and busy-period distributions in M/G/1 queues are obtained via Laplace–Stieltjes transforms (MGF-type objects), as in the Pollaczek–Khinchine formula.

Option pricing (finance). The Carr–Madan method and the Heston stochastic-volatility model price options by Fourier-inverting the characteristic function of the log-asset price, which is known in closed form even when the density is not.

Signal processing and physics. The CF of a noise distribution is its spectral fingerprint; in physics the CF of position/momentum is central to characteristic-function methods in quantum optics (the Wigner and characteristic functions of states).

Statistical estimation. Empirical characteristic function methods estimate parameters of distributions (e.g., stable distributions) for which the density has no closed form but the CF does.

Reliability engineering. Sums and minima of lifetimes are analyzed through transforms to obtain system failure-time distributions from component distributions.

Historical Development

1810s — Laplace. Pierre-Simon Laplace uses what are effectively generating-function and transform techniques in his probability work, including early versions of the central limit phenomenon.
1853 — Cauchy. Augustin-Louis Cauchy studies functions of the form $E [e^{i tX}]$ and the distribution now bearing his name, an early example of a law with no mean and no MGF.
1900–1901 — Lyapunov. Aleksandr Lyapunov gives the first rigorous proof of the Central Limit Theorem under his condition, using characteristic functions as the key tool.
1920s — Lévy. Paul Lévy develops the systematic theory: the inversion formula, the continuity theorem, and the role of CFs in classifying stable and infinitely divisible laws.
1922 — Pólya. George Pólya proves criteria (Pólya's theorem) giving sufficient conditions for a function to be a characteristic function.
1932–1934 — Cramér & Bochner. Harald Cramér advances analytic methods; Bochner's theorem characterizes exactly which functions are characteristic functions (continuous, positive-definite, value 1 at 0).
1935 — Lévy–Cramér theorem. If $X + Y$ is normal with $X, Y$ independent, then both $X$ and $Y$ are normal — a striking decomposition result proved via CFs.
1937 — Khinchin. Aleksandr Khinchin systematizes infinitely divisible distributions, completing the Lévy–Khinchin representation of their characteristic functions.
Mid–20th century onward. Characteristic-function and transform methods become standard in textbooks (Feller, Loève, Billingsley) and the default rigorous tool in modern probability.

Subtleties & Limits

The moment problem. Given a sequence

{m_{n}}

, when is it the moment sequence of some distribution, and is that distribution unique? The Hamburger, Stieltjes, and Hausdorff moment problems address existence on

R

[0, \infty)

, and

[0, 1]

respectively. Uniqueness (determinacy) is delicate: the Carleman condition

\sum_{n} m_{2 n}^{- 1/2 n} = \infty

is sufficient but not necessary, and there is no simple necessary-and-sufficient closed criterion. Constructing explicit non-uniqueness families (as for the log-normal) remains an active area. Reconstructing the distribution from a truncated transform. In practice one knows the CF only on a bounded range or with noise. Stable numerical inversion (regularized Fourier inversion, the Gil-Pelaez formula in practice) is genuinely hard for heavy-tailed or oscillatory CFs, and quantifying optimal recovery rates is an ongoing statistical question. Characterizing characteristic functions. Bochner's theorem describes CFs abstractly (positive-definite functions), but deciding whether a given explicit function is a CF — and reconstructing the law efficiently — can be subtle, with Pólya-type sufficient conditions filling part of the gap.