Random Variables and Characteristic Functions

The characteristic function of a random variable $X$ is $φ_{X} (t) = E [e^{i tX}]$ — the Fourier transform of its probability distribution. Key result: the characteristic function always exists, uniquely determines the distribution, and converts convolution into multiplication, making it the master tool for sums of independent variables, moment extraction, and proofs of the Central Limit Theorem. Applies wherever probability distributions, sums of random effects, or limit theorems appear. Introduced by Lyapunov and Lévy (early 20th century), built on Fourier (1820s) and refined by the Russian school.

A random variable is a measurable function from a probability space to the real numbers, assigning a numerical outcome to each elementary event. Its distribution captures all probabilistic information, but distributions are awkward to manipulate directly — especially under addition. The characteristic function $φ_{X} (t) = E [e^{i tX}]$ encodes the same information as a complex-valued function of a real parameter $t$ , and because $∣ e^{i tX} ∣ = 1$ it exists for every random variable, unlike the moment generating function. The surprising fact: from a single transform you can recover the entire distribution, read off every moment, prove independence of sums collapses into a product, and derive the bell curve as a universal limit — all by elementary calculus on $φ$ .

The Core Insight

Multiplication is easier than convolution. Adding independent random variables convolves their densities — a messy integral — but multiplies their characteristic functions. By translating probability into the language of complex exponentials, hard analytic operations become arithmetic, and the unique correspondence between $φ_{X}$ and the distribution guarantees nothing is lost in translation.

Formal Statement

Random Variable

Given a probability space $(Ω, F, P)$ , a random variable is a measurable function $X : Ω \to R$ , meaning ${ω : X (ω) \leq x} \in F$ for every $x \in R$ . Its cumulative distribution function (CDF) is

F_{X} (x) = P (X \leq x) = P ({ω : X (ω) \leq x}) .

⟳directly evaluate the probability measure on the preimage of

(- \infty, x]

; measurability guarantees this set lies in

F

If $F_{X}$ is absolutely continuous, there is a probability density function (PDF) $f_{X}$ with $F_{X} (x) = \int_{- \infty}^{x} f_{X} (u) d u$ . The expectation of a function $g (X)$ is

E [g (X)] = \int_{- \infty}^{\infty} g (x) d F_{X} (x) = \int_{- \infty}^{\infty} g (x) f_{X} (x) d x (continuous case) .

⟳the law of the unconscious statistician — integrate

g

against the pushforward measure

P \circ X^{- 1}

, written as a Lebesgue–Stieltjes integral against

F_{X}

Characteristic Function

The characteristic function (CF) of a random variable $X$ is the function $φ_{X} : R \to C$ defined by

φ_{X} (t) = E [e^{i tX}] = \int_{- \infty}^{\infty} e^{i t x} d F_{X} (x),

⟳apply [2] with

g (x) = e^{i t x} = cos t x + i sin t x

, splitting into real and imaginary integrable parts since

∣ cos ∣, ∣ sin ∣ \leq 1

where $t \in R$ and $i = - 1$ . Symbols: $φ_{X}$ is complex-valued; $t$ is a real parameter (the "frequency"); $d F_{X}$ denotes integration against the distribution (sum over a probability mass function in the discrete case).

Fundamental Properties

For any random variable $X$ :

φ_{X} (0) = 1, ∣ φ_{X} (t) ∣ \leq 1, \overline{φ_{X} (t)} = φ_{X} (- t),

⟳

φ_{X} (0) = E [1] = 1

;

∣ φ_{X} (t) ∣ = ∣ E [e^{i tX}] ∣ \leq E ∣ e^{i tX} ∣ = 1

by Jensen/triangle inequality; conjugate

e^{i t x}

and $φ_{X}$ is uniformly continuous on $R$ . For an affine transform $Y = a X + b$ ,

φ_{a X + b} (t) = e^{i t b} φ_{X} (a t) .

⟳

E [e^{i t (a X + b)}] = e^{i t b} E [e^{i (t a) X}]

, factoring the deterministic exponential out of the expectation.

Convolution / Sum of Independents

If $X$ and $Y$ are independent random variables, then for $S = X + Y$ :

φ_{X + Y} (t) = φ_{X} (t) φ_{Y} (t) .

⟳

E [e^{i t (X + Y)}] = E [e^{i tX} e^{i t Y}] = E [e^{i tX}] E [e^{i t Y}]

, where independence factors the expectation of the product.

Moment Generation

If $E ∣ X ∣^{n} < \infty$ , then $φ_{X}$ is $n$ -times differentiable and

E [X^{n}] = \frac{1}{i ^{n}} φ_{X}^{(n)} (0) = (- i)^{n} φ_{X}^{(n)} (0) .

⟳differentiate under the integral sign

n

times, bringing down a factor

(i x)^{n}

, then set

t = 0

; integrability of

∣ X ∣^{n}

justifies the interchange via dominated convergence.

Inversion and Uniqueness

The characteristic function determines the distribution uniquely: if $φ_{X} = φ_{Y}$ then $F_{X} = F_{Y}$ . When $φ_{X} \in L^{1} (R)$ , the density exists and is recovered by

f_{X} (x) = \frac{1}{2 π} \int_{- \infty}^{\infty} e^{- i t x} φ_{X} (t) d t .

⟳this is the inverse Fourier transform; substitute

φ_{X} (t) = \int e^{i t y} f_{X} (y) d y

and use

\frac{1}{2 π} \int e^{i t (y - x)} d t = δ (y - x)

as a distributional identity (rigorously via the Lévy inversion formula with a Gaussian damping limit).

Uniqueness via Lévy's Inversion Formula

We show $φ$ determines $F$ at continuity points.

Step 1. For real

a < b

, consider the integral

I_{T} = \frac{1}{2 π} \int_{- T}^{T} \frac{e ^{- i t a} - e ^{- i t b}}{i t} φ_{X} (t) d t .

Step 2. Insert

φ_{X} (t) = \int e^{i t x} d F_{X} (x)

and exchange the order of integration (justified by Fubini since the integrand is bounded on

[- T, T] \times R

I_{T} = \int_{- \infty}^{\infty} [\frac{1}{2 π} \int_{- T}^{T} \frac{e ^{i t (x - a)} - e ^{i t (x - b)}}{i t} d t] d F_{X} (x) .

Step 3. The inner bracket is a Dirichlet-type kernel; using

\int_{- T}^{T} \frac{s i n ( θ t )}{t} d t \to π sgn (θ)

T \to \infty

, the bracket converges to

\frac{1}{2} [1_{x > a} - 1_{x > b}]

plus boundary terms. Step 4. Taking

T \to \infty

yields

T \to \infty lim I_{T} = F_{X} (b) - F_{X} (a)

at continuity points

a, b

. Hence the increments of

F_{X}

— and so

F_{X}

itself — are fully determined by

φ_{X}

■

:::svg[Characteristic function of a standard normal: $φ (t) = e^{- t^{2} /2}$ traced as the real-axis projection of $E [e^{i tX}]$ ]

<svg viewBox="0 0 460 280" xmlns="http://www.w3.org/2000/svg">
<rect width="460" height="280" fill="#0a0e17"/>
<line x1="40" y1="200" x2="440" y2="200" stroke="#94a3b8" stroke-width="1.2"/>
<line x1="240" y1="40" x2="240" y2="220" stroke="#94a3b8" stroke-width="1.2"/>
<text x="445" y="204" fill="#e2e8f0" font-size="12">t</text>
<text x="248" y="50" fill="#e2e8f0" font-size="12">φ(t)</text>
<text x="232" y="216" fill="#e2e8f0" font-size="11">0</text>
<text x="226" y="60" fill="#e2e8f0" font-size="11">1</text>
<line x1="236" y1="65" x2="244" y2="65" stroke="#94a3b8" stroke-width="1"/>
<path d="M40,199 C100,196 160,160 240,65 C320,160 380,196 440,199"
      fill="none" stroke="#d4a853" stroke-width="2.4"/>
<text x="300" y="120" fill="#d4a853" font-size="13">φ(t)=e^(−t²/2)</text>
<text x="60" y="240" fill="#e2e8f0" font-size="11">Real, even, φ(0)=1, |φ|≤1, decays to 0</text>
</svg>

Paul Lévy | 1886–1971

The French mathematician Paul Lévy was the architect of the modern theory of characteristic functions. In his 1925 Calcul des probabilités he proved the inversion formula bearing his name and the continuity theorem linking pointwise convergence of characteristic functions to convergence in distribution — the engine of nearly every modern proof of the Central Limit Theorem. Lévy also pioneered stochastic processes (Lévy processes, Brownian motion fine structure), stable laws, and the concept of infinitely divisible distributions, for which characteristic functions are the natural language via the Lévy–Khintchine formula.

Building Intuition

Think of a distribution as a "shape" of mass spread along the real line. The characteristic function is its fingerprint in frequency space: $φ_{X} (t)$ measures how much the distribution "resonates" with a wave $e^{i t x}$ of frequency $t$ . At $t = 0$ all the mass is counted in phase, giving $φ (0) = 1$ . As $t$ grows, the wave oscillates faster; if the distribution is smooth and concentrated, contributions from nearby mass start cancelling, so $φ$ shrinks — narrow distributions have wide fingerprints and vice versa (the uncertainty principle of Fourier analysis). The magic of $∣ e^{i tX} ∣ = 1$ is that no matter how heavy the tails — even for a Cauchy variable with no mean — the wave never blows up, so the fingerprint always exists. Sums of independent variables stack their wave-distortions multiplicatively, which is why adding randomness is a multiplication in fingerprint space.

Worked Examples

Sum of Two Independent Poissons

Given:

X \sim Poisson (λ_{1})

and

Y \sim Poisson (λ_{2})

, independent, with

λ_{1} = 3

λ_{2} = 5

. Find the distribution of

S = X + Y

. Step 1 — CF of a Poisson. For

X \sim Poisson (λ)

φ_{X} (t) = k = 0 \sum \infty e^{i t k} \frac{e ^{- λ} λ ^{k}}{k !} = e^{- λ} k = 0 \sum \infty \frac{( λ e ^{i t} ) ^{k}}{k !} = e^{λ (e^{i t} - 1)} .

Step 2 — Multiply (independence, [6]).

φ_{S} (t) = e^{λ_{1} (e^{i t} - 1)} \cdot e^{λ_{2} (e^{i t} - 1)} = e^{(λ_{1} + λ_{2}) (e^{i t} - 1)} .

Step 3 — Recognize the form. This is exactly the CF of a Poisson with parameter

λ_{1} + λ_{2} = 8

. By uniqueness ([8]),

S \sim Poisson (8), P (S = k) = \frac{e ^{- 8} 8 ^{k}}{k !}

The Poisson family is closed under independent addition — proven in three lines with characteristic functions instead of a convolution sum.

Mean and Variance of the Exponential via Differentiation

Given:

X \sim Exponential (λ)

with density

f_{X} (x) = λ e^{- λ x}

x \geq 0

, and

λ = 2

. Find

E [X]

and

Var (X)

. Step 1 — Characteristic function.

φ_{X} (t) = \int_{0}^{\infty} e^{i t x} λ e^{- λ x} d x = λ \int_{0}^{\infty} e^{- (λ - i t) x} d x = \frac{λ}{λ - i t} .

Step 2 — First derivative ([7], $n = 1$ ).

φ_{X}^{'} (t) = \frac{λ i}{( λ - i t ) ^{2}}, φ_{X}^{'} (0) = \frac{λi}{λ ^{2}} = \frac{i}{λ} .

E [X] = \frac{1}{i} φ_{X}^{'} (0) = \frac{1}{λ} = \frac{1}{2} = 0.5

. Step 3 — Second derivative.

φ_{X}^{''} (t) = \frac{- 2 λ}{( λ - i t ) ^{3}}, φ_{X}^{''} (0) = \frac{- 2 λ}{λ ^{3}} = \frac{- 2}{λ ^{2}} .

E [X^{2}] = (- i)^{2} φ_{X}^{''} (0) = - φ_{X}^{''} (0) = \frac{2}{λ ^{2}} = \frac{2}{4} = 0.5

. Step 4 — Variance.

Var (X) = E [X^{2}] - (E [X])^{2} = \frac{2}{λ ^{2}} - \frac{1}{λ ^{2}} = \frac{1}{λ ^{2}} .

E [X] = 0.5, Var (X) = 0.25

(in the units of

X

; variance in squared units).

Common Misconceptions

The characteristic function is not the moment generating function (MGF). The MGF is

M_{X} (t) = E [e^{tX}]

(no

i

), and it can fail to exist for heavy-tailed distributions because

e^{tX}

may have infinite expectation. The characteristic function

E [e^{i tX}]

always exists because

∣ e^{i tX} ∣ = 1

. When the MGF exists in a neighborhood of

0

, the two are related by

φ_{X} (t) = M_{X} (i t)

. Never assume an MGF exists; the CF is the safe universal tool. A real, symmetric distribution has a real-valued CF — but the converse logic is subtle.

φ_{X}

is real-valued iff

X

is symmetric about

0

(i.e.

X

and

- X

have the same distribution). Students often forget that a general

φ_{X} (t)

is genuinely complex. Also,

φ_{X}

being differentiable at

0

does not guarantee a finite mean unless you have the stronger two-sided condition; differentiability of

φ

and existence of moments must be handled carefully.

The Cauchy distribution has no mean — and its CF reveals why

The standard Cauchy distribution has density $f (x) = \frac{1}{π ( 1 + x ^{2} )}$ and characteristic function

φ_{X} (t) = e^{- ∣ t ∣} .

This function is continuous and

∣ φ_{X} ∣ \leq 1

as required, but it is not differentiable at $t = 0$ (the left and right derivatives are

+ 1

and

- 1

). By [7], this non-differentiability signals that

E [X]

does not exist. Indeed

\int ∣ x ∣ f (x) d x = \infty

. Moreover, if

X_{1}, \dots, X_{n}

are iid Cauchy, then

φ_{\overset{ˉ}{X}} (t) = (e^{- ∣ t / n ∣})^{n} = e^{- ∣ t ∣}

— the sample mean of $n$ Cauchy variables is again standard Cauchy, so averaging never reduces the spread. The Law of Large Numbers fails, and the CF makes this transparent.

Connections

Characteristic functions sit at the crossroads of analysis and probability. They are the Fourier Transform of the distribution, inheriting inversion, convolution, and Plancherel theorems wholesale. Pointwise convergence of $φ_{n} \to φ$ (with $φ$ continuous at $0$ ) is equivalent to convergence in distribution by Lévy's Continuity Theorem, which is the standard route to the Central Limit Theorem. The class of CFs that are limits of CFs of sums forms the infinitely divisible distributions, classified by the Lévy–Khintchine Formula. For random vectors, the multivariate CF $φ_{X} (t) = E [e^{i t \cdot X}]$ encodes the joint law and detects independence. The cousin generating tools — Moment Generating Function and Probability Generating Function — are special evaluations valid under stronger conditions.

Real-World Applications

Signal processing & noise modeling. Summing independent noise sources multiplies their characteristic functions; Gaussian thermal noise is the universal limit predicted by the CLT, justifying additive Gaussian models in communications.
Insurance & actuarial risk (Panjer/aggregate claims). Total claim amount $S = \sum_{i = 1}^{N} X_{i}$ with random $N$ has CF $φ_{S} (t) = G_{N} (φ_{X} (t))$ ; Fourier inversion (FFT) recovers the aggregate loss distribution for setting premiums.
Option pricing (Carr–Madan formula). Modern financial engineering prices European options by Fourier inversion of the characteristic function of the log-asset price, central to the Heston and variance-gamma models where densities have no closed form.
Statistical physics — random walks & diffusion. The position after $n$ independent steps has a CF that is a power of the single-step CF; its small- $t$ expansion yields the diffusion (heat) equation and the Gaussian propagator.
Goodness-of-fit testing. The empirical characteristic function $\overset{φ}{^} (t) = \frac{1}{n} \sum e^{i t X_{j}}$ underlies tests of normality and parameter estimation when likelihoods are intractable (e.g., stable distributions).
Quantum mechanics analogy. The Wigner characteristic function and moment expansions mirror this machinery; expectation values of $e^{i ξ \overset{x}{^}}$ generate the position distribution.
Telecommunications — error analysis. Bit-error rates for digitally modulated signals over fading channels are computed by characteristic-function methods (MGF approach to SNR averaging).

Historical Development

1733–1812 — Foundations of the normal law. De Moivre, Laplace, and Gauss develop the bell curve; Laplace uses transform methods (the "generating function") foreshadowing characteristic functions.
1822 — Fourier. Joseph Fourier publishes the Théorie analytique de la chaleur, establishing the Fourier transform that characteristic functions specialize.
1900s — Lyapunov. Aleksandr Lyapunov gives a rigorous proof of the Central Limit Theorem using characteristic-function (transform) techniques, the first modern use in probability.
1920s — Lévy. Paul Lévy systematizes characteristic functions in Calcul des probabilités (1925), proving the inversion formula and continuity theorem.
1933 — Kolmogorov. Kolmogorov's measure-theoretic axiomatization of probability gives random variables and their distributions a rigorous foundation, legitimizing $E [e^{i tX}]$ .
1934–1937 — Khintchine & Lévy. The Lévy–Khintchine formula classifies infinitely divisible distributions via their characteristic functions.
1936 — Bochner's theorem. Salomon Bochner characterizes exactly which functions are characteristic functions (continuous, positive-definite, value 1 at 0).
1937 — Cramér. Harald Cramér's Random Variables and Probability Distributions consolidates the theory for a generation of statisticians.
1990s — Computational era. Carr–Madan (1999) and the rise of FFT-based inversion make characteristic functions a practical engine in quantitative finance and computational statistics.

Subtleties & Limits

Several deep questions surround characteristic functions. Bochner's theorem tells us which functions are CFs (continuous, positive-definite, $φ (0) = 1$ ), but verifying positive-definiteness for a candidate function is often hard; constructing distributions with prescribed properties via their CFs remains delicate. The relationship between smoothness of $φ$ and existence of moments is only partly characterized: $2 n$ -fold differentiability of $φ$ at $0$ implies finite $2 n$ -th moments, but the odd-order and one-sided cases hide pathologies. Decay rates of $φ$ at infinity govern the smoothness of the density (a faster-decaying CF means a smoother density), tying into open questions in harmonic analysis about which decay/regularity pairings are achievable. Finally, in high dimensions and for dependent structures, recovering and estimating multivariate distributions from empirical characteristic functions faces curse-of-dimensionality and stability issues that remain active research in modern statistics.