Convolution Theorem and Applications

One-line statement: The Fourier transform of a convolution of two functions equals the (pointwise) product of their individual Fourier transforms, and vice versa. Key result in words: Convolution in one domain (time/space) becomes multiplication in the other domain (frequency). This turns hard integral operations into simple multiplications. Where it applies: Signal processing, linear systems, probability (sums of random variables), optics, image processing, differential equations, acoustics. Who/when: Rooted in the work of Joseph Fourier (1807–1822); the convolution operation and its transform property were formalized through the 19th and 20th centuries, becoming central to engineering with the Fast Fourier Transform (Cooley–Tukey, 1965).

The Convolution Theorem states that for suitable functions $f$ and $g$ , the Fourier transform of their convolution $(f * g)$ is the product of their transforms: $F {f * g} = F {f} \cdot F {g}$ . Convolution itself is the integral that describes how the shape of one function is "smeared" by another — it is the natural operation governing every linear time-invariant (LTI) system, where the output equals the input convolved with the system's impulse response. The theorem matters because convolution integrals are computationally expensive and analytically awkward, while multiplication is trivial; moving to the frequency domain converts one into the other. The surprising fact: a single mathematical identity simultaneously explains why blurred photographs can be sharpened, why the sum of two dice has a triangular distribution, and why fast polynomial multiplication is possible — all are instances of the same theorem.

The Core Insight

A linear system is completely characterized by how it responds to a single impulse. Any input can be decomposed into a continuum of scaled, shifted impulses, and by linearity and time-invariance, the output is the superposition of shifted impulse responses — this superposition is convolution. The Fourier transform diagonalizes this operation because complex exponentials $e^{iω t}$ are the eigenfunctions of every LTI system: feeding in a pure frequency yields the same frequency scaled by a complex number. Convolution, which mixes everything in the time domain, becomes independent scaling of each frequency — i.e., pointwise multiplication.

Formal Statement

Convolution

The convolution of two functions $f, g : R \to C$ is defined by

(f * g) (t) = \int_{- \infty}^{\infty} f (τ) g (t - τ) d τ

⟳For each output time

t

, flip

g

about the origin, shift it by

t

, multiply by

f

, and integrate over all

τ

. The variable

τ

slides

g

across

f

Here $t$ is the output (time or space) variable, $τ$ is the integration (dummy) variable, $f$ and $g$ are the two functions being combined, and the result $(f * g)$ is itself a function of $t$ . Convolution is commutative: $f * g = g * f$ .

Fourier Transform Convention

We adopt the angular-frequency convention:

\hat{f} (ω) = F {f} (ω) = \int_{- \infty}^{\infty} f (t) e^{- iω t} d t

⟳Project

f

onto the basis of complex exponentials;

\hat{f} (ω)

is the "amount" of frequency

ω

present in

f

where $ω$ is angular frequency (rad/s), $i = - 1$ , and $\hat{f}$ is the transform. The inverse is $f (t) = \frac{1}{2 π} \int \hat{f} (ω) e^{iω t} d ω$ .

Convolution Theorem

Let $f, g \in L^{1} (R)$ . Then their convolution is also integrable, and

F {f * g} (ω) = \hat{f} (ω) \overset{g}{^} (ω)

⟳Substitute the definition of convolution into the transform integral and exchange the order of integration; the shift produces a factored exponential.

The dual (modulation) form states that pointwise multiplication in time corresponds to convolution in frequency (up to a $2 π$ factor):

F {f \cdot g} (ω) = \frac{1}{2 π} (\hat{f} * \overset{g}{^}) (ω)

⟳Apply the theorem in reverse using the inverse transform, where the

1/2 π

from the inverse convention carries through.

Why It Holds / Derivation

Proof of the Convolution Theorem

We compute the Fourier transform of $(f * g)$ directly.

Step 1. Write the transform of the convolution:

F {f * g} (ω) = \int_{- \infty}^{\infty} [\int_{- \infty}^{\infty} f (τ) g (t - τ) d τ] e^{- iω t} d t

⟳Insert equation [1] into equation [2].

Step 2. Because

f, g \in L^{1}

, the double integral converges absolutely, so by Fubini's theorem we swap the order of integration:

= \int_{- \infty}^{\infty} f (τ) [\int_{- \infty}^{\infty} g (t - τ) e^{- iω t} d t] d τ

⟳Pull

f (τ)

out of the inner integral since it does not depend on

t

Step 3. Substitute

u = t - τ

(so

t = u + τ

d t = d u

) in the inner integral:

\int_{- \infty}^{\infty} g (u) e^{- iω (u + τ)} d u = e^{- iω τ} \int_{- \infty}^{\infty} g (u) e^{- iω u} d u = e^{- iω τ} \overset{g}{^} (ω)

⟳The exponential splits:

e^{- iω (u + τ)} = e^{- iω τ} e^{- iω u}

; the

τ

factor is constant w.r.t.

u

Step 4. Substitute back:

F {f * g} (ω) = \int_{- \infty}^{\infty} f (τ) e^{- iω τ} \overset{g}{^} (ω) d τ = \overset{g}{^} (ω) \int_{- \infty}^{\infty} f (τ) e^{- iω τ} d τ = \hat{f} (ω) \overset{g}{^} (ω)

⟳

\overset{g}{^} (ω)

is constant in

τ

and factors out, leaving exactly the transform of

f

■

Joseph Fourier | 1768–1830

Jean-Baptiste Joseph Fourier introduced, in his 1822 Théorie analytique de la chaleur, the radical claim that any function could be represented as a sum (or integral) of sines and cosines. While studying heat diffusion, he developed the transform machinery from which the convolution theorem flows. His ideas were initially resisted by Lagrange and Laplace over questions of convergence, but they became one of the most powerful tools in all of applied mathematics. The decomposition of signals into frequencies — and the consequent simplification of convolution — is his enduring legacy.

Building Intuition

Think of convolution as weighted blurring. If $g$ is a narrow bump (a "kernel") and $f$ is a signal, then $(f * g)$ replaces each point of $f$ with a $g$ -shaped average of its neighbors. A wide $g$ blurs heavily; a delta-spike $g$ leaves $f$ unchanged ( $f * δ = f$ ).

Now imagine each frequency component of $f$ as an independent "channel." An LTI system can only scale and phase-shift each channel — it cannot create new frequencies. The number it multiplies each channel by is precisely $\overset{g}{^} (ω)$ , the frequency response. So blurring in time = attenuating high frequencies, which is exactly multiplication by a $\overset{g}{^}$ that decays at high $ω$ . The theorem says: the messy overlap-and-integrate dance in time is just dial-turning in frequency.

Convolution as flip-shift-multiply-integrate: the kernel $g$ slides across $f$, and the shaded overlap area produces one output value.

Worked Examples

Two Rectangular Pulses → Triangle

Given: Let

f (t) = g (t) = rect (t)

, the unit rectangular pulse equal to

1

for

- \frac{1}{2} \leq t \leq \frac{1}{2}

and

0

otherwise. Find:

(f * g) (t)

, verified two ways. Direct (time domain): The convolution of two unit-width rectangles is a triangle of base width

2

and peak height

1

(f * g) (t) = tri (t) = {1 - ∣ t ∣, 0, ∣ t ∣ \leq 1 ∣ t ∣ > 1

⟳The overlap of two unit rectangles, one shifted by

t

, has length

1 - ∣ t ∣

for

∣ t ∣ < 1

Frequency check: The transform of

rect

\hat{f} (ω) = sinc (ω /2) = \frac{sin ( ω /2 )}{ω /2}

. By the theorem the transform of the convolution is

(f * g) (ω) = \hat{f} (ω) \overset{g}{^} (ω) = (\frac{sin ( ω /2 )}{ω /2})^{2} = sinc^{2} (ω /2)

⟳Square the sinc — and indeed

sinc^{2}

is the known transform of the triangle, confirming the result.

Boxed answer:

(f * g) (t) = tri (t), (f * g) (ω) = sinc^{2} (ω /2)

Sum of Two Dice via Convolution of PMFs

Given: Two fair six-sided dice. Each die's outcome has probability mass function (PMF)

p (k) = \frac{1}{6}

for

k = 1, \dots, 6

. The PMF of the sum

S

of two independent dice is the discrete convolution

p * p

. Compute: The discrete convolution

(p * p) (n) = k \sum p (k) p (n - k) = \frac{1}{36} (number of ways two dice sum to n)

⟳Independence makes the joint PMF a product; summing over all

(k, n - k)

pairs counts the favorable outcomes.

Evaluating for $n = 2, \dots, 12$ gives counts $1, 2, 3, 4, 5, 6, 5, 4, 3, 2, 1$ , hence:

| Sum $n$ | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | |---|---|---|---|---|---|---|---|---|---|---|---| | $P (S = n)$ | 1/36 | 2/36 | 3/36 | 4/36 | 5/36 | 6/36 | 5/36 | 4/36 | 3/36 | 2/36 | 1/36 |

This is the discrete triangular distribution — the same triangle shape as Example 1, reflecting that adding independent variables convolves their distributions.

Boxed answer:

P (S = 7) = \frac{6}{36} = \frac{1}{6} \approx 0.167 (the most probable sum)

Probabilistic note: Taking the characteristic function (the Fourier transform of the PMF) turns this convolution into the product

φ_{S} (t) = φ (t)^{2}

— the engine behind much of probability theory.

Common Misconceptions

Convolution is not multiplication in the same domain

A frequent error is to compute $(f * g) (t)$ as $f (t) g (t)$ . These are entirely different operations. Multiplication in the time domain corresponds to convolution in the frequency domain (equation [4]), not the reverse. Confusing the two leads, for instance, to wrong filter outputs. Also note the $2 π$ factor in the dual form — it depends on your transform convention. With the symmetric ( $1/ 2 π$ ) convention or the ordinary-frequency convention ( $e^{- 2 π i f t}$ ), the constants move around. Always state your convention before quoting the theorem.

The theorem can fail without integrability/decay assumptions

The theorem in form [3] requires $f, g \in L^{1}$ so the convolution exists and Fubini applies. Consider $f (t) = g (t) = 1$ (constant functions). Their convolution $\int_{- \infty}^{\infty} 1 \cdot 1 d τ$ diverges — it does not exist as an ordinary function. Meanwhile their Fourier transforms are $\hat{f} = \overset{g}{^} = 2 π δ (ω)$ , and the naïve product $(2 π)^{2} δ (ω)^{2}$ is undefined (the square of a Dirac delta has no meaning). The clean identity holds only when both sides are well-defined; extending it to such cases requires the theory of tempered distributions and care about which products are admissible.

Connections

The convolution theorem is the hub linking several areas. In Linear Time-Invariant Systems, the output is $y = x * h$ where $h$ is the impulse response, so $\overset{y}{^} = \overset{x}{^} \hat{h}$ — defining the transfer function $\hat{h} (ω)$ . In Laplace Transform, the analogous property $L {f * g} = F (s) G (s)$ governs circuit and control analysis. In Probability Theory, the density of a sum of independent random variables is the convolution of their densities, transformed via the characteristic function. In Z-Transform, discrete convolution of sequences maps to multiplication, foundational to digital filters. The theorem also underlies the Cross-Correlation and autocorrelation operations through the Wiener–Khinchin theorem, which ties autocorrelation to the power spectral density.

Real-World Applications

Concrete Uses

Fast polynomial & integer multiplication. Polynomial coefficients convolve when polynomials multiply; using the FFT, multiplication of degree- $n$ polynomials drops from $O (n^{2})$ to $O (n lo g n)$ . The Schönhage–Strassen algorithm multiplies huge integers this way.

Digital filtering (audio/EQ). An audio equalizer applies a filter kernel $h$ ; output $= x * h$ . Engineers design $\hat{h} (ω)$ in frequency (boost bass, cut treble), then realize it as convolution — or do the convolution directly via FFT for long kernels (fast convolution / overlap-add).

Image blurring and sharpening. A Gaussian blur convolves the image with a Gaussian kernel; deconvolution (dividing by $\overset{g}{^}$ in frequency) reverses motion blur or out-of-focus blur, used in forensic and medical imaging.

Convolutional Neural Networks (CNNs). Each convolutional layer slides learned kernels across feature maps; this is discrete 2-D convolution, the operation that gives CNNs translation-equivariance for image recognition.

Optics and diffraction. The image formed by a lens is the object convolved with the point spread function (PSF). The optical transfer function $= \hat{PSF}$ quantifies resolution — directly the convolution theorem.

Seismology and reflection imaging. A seismic trace models as the earth's reflectivity convolved with a source wavelet; deconvolution recovers reflectivity to map subsurface layers for oil exploration.

GPS and radar (matched filtering). Detecting a known pulse in noise uses correlation (a convolution with a time-reversed template); the FFT makes real-time detection feasible.

Historical Development

1754 — d'Alembert uses an integral resembling convolution in a Taylor-series remainder, an early appearance of the operation.
1807 — Fourier submits his memoir on heat to the French Academy, proposing trigonometric-series decompositions; the seed of frequency-domain analysis.
1822 — Fourier publishes Théorie analytique de la chaleur, establishing the Fourier integral and transform.
1880s — Wiener, Heaviside (operational calculus) develop operator methods that implicitly use convolution-as-multiplication for solving differential equations in engineering.
1934 — Norbert Wiener formalizes generalized harmonic analysis; the Wiener–Khinchin theorem links autocorrelation and spectrum.
1940s — Claude Shannon & engineers apply convolution rigorously to communication and sampling theory; LTI-system theory matures.
1947 — Laurent Schwartz founds the theory of distributions, giving rigorous meaning to convolutions of objects like the Dirac delta.
1965 — Cooley & Tukey publish the Fast Fourier Transform, reducing the DFT to $O (n lo g n)$ and making fast convolution practical — revolutionizing digital signal processing.
1980s–present — Convolution becomes the computational backbone of digital imaging, MP3/JPEG codecs, and (2010s) deep learning via CNNs.

Subtleties & Limits

Boundaries of the Theorem

The clean identity assumes enough regularity ( $L^{1}$ , or appropriate distributional setting). Several genuine subtleties remain instructive at and beyond the undergraduate level:

Existence of $f * g$ . For $f \in L^{p}$ and $g \in L^{q}$ , Young's convolution inequality guarantees $f * g \in L^{r}$ with $\frac{1}{r} = \frac{1}{p} + \frac{1}{q} - 1$ ; the sharp constants in this inequality (Beckner, Brascamp–Lieb) connect to deep geometric-analytic questions.
Products of distributions. Multiplying two distributions (the dual side of convolution) is generally ill-defined; Hörmander's wavefront-set criterion specifies when it works — central to quantum field theory's renormalization difficulties.
Circular vs. linear convolution. The DFT computes circular convolution; recovering linear convolution requires zero-padding. Off-by-one and aliasing errors here are perennial practical pitfalls.
Deconvolution ill-posedness. Dividing by $\overset{g}{^} (ω)$ to invert a blur amplifies noise wherever $\overset{g}{^}$ is small — an inverse problem requiring regularization (Tikhonov, Wiener filtering); robust blind deconvolution remains an active research area.