Symmetric Kernels and Mercer's Theorem

An integral operator $K$ smears a function against a two-variable kernel, $(K\phi)(x)=\int_a^b K(x,y)\phi(y)\,\mathrm{d}y$ . Read the kernel as an infinite continuous matrix indexed by $(x,y)$ , and one symmetry towers over the rest: when $K(y,x)=\overline{K(x,y)}$ the operator is Hermitian — self-adjoint — on $L^2(a,b)$ . Everything this article proves flows from that single algebraic fact, exactly as the spectral theory of Hermitian matrices flows from $A=A^{\dagger}$ . Self-adjointness forces the eigenvalues real and the eigenfunctions orthogonal; square-integrability makes the operator compact, so the spectrum is discrete and can pile up only at zero; and the operator is then diagonalized by an orthonormal eigenbasis in which every integral equation decouples into scalar equations. Two theorems set the frame. The Hilbert–Schmidt theorem handles any square-integrable symmetric kernel and delivers the eigenexpansion $K(x,y)\sim\sum_n\lambda_n\phi_n(x)\overline{\phi_n(y)}$ — but only in mean square, saying nothing about any particular point. Mercer's theorem asks the sharper question: when does that bilinear series converge everywhere, absolutely and uniformly, so that you may legally set $x=y$ ? The answer needs two extra hypotheses — continuity on the compact square and positive semi-definiteness — and its reward is the trace identity $\sum_n\lambda_n=\int_a^b K(x,x)\,\mathrm{d}x$ . This is the functional-analytic backbone of Sturm–Liouville Green's functions and Fredholm theory, and — in modern dress — of covariance operators, the Karhunen–Loève expansion, and the kernel machines of machine learning.

Key takeaways

A Hermitian kernel $K(y,x)=\overline{K(x,y)}$ makes the integral operator self-adjoint on $L^2(a,b)$ , so its spectrum is real and eigenfunctions orthonormal — the continuous analogue of diagonalizing a Hermitian matrix.
Square-integrability $\iint|K|^2<\infty$ makes the operator compact (Hilbert–Schmidt): the spectrum is discrete with $\sum_n\lambda_n^2<\infty$ and $\lambda_n\to0$ , and the kernel expansion converges in mean square.
Mercer's theorem adds continuity and positivity ( $\lambda_n\ge0$ ): then $K(x,y)=\sum_n\lambda_n\phi_n(x)\overline{\phi_n(y)}$ converges absolutely and uniformly, so it is safe to evaluate on the diagonal.
Diagonalizing turns the Fredholm equation $(I-\lambda K)\phi=f$ into decoupled scalar equations, with resonances at the characteristic values $\lambda=1/\lambda_n$ where the Fredholm alternative decides solvability.
The trace formula $\sum_n\lambda_n=\int_a^b K(x,x)\,\mathrm{d}x$ follows on the diagonal; for the Brownian kernel $\min(x,y)$ on $[0,1]$ both sides equal exactly $1/2$ .
Key idea: a symmetric kernel makes the integral operator self-adjoint, compactness forces a discrete real spectrum with orthonormal eigenfunctions, and Mercer's continuity-plus-positivity upgrades the eigenexpansion of the kernel from $L^2$ convergence to absolute uniform convergence.

The core insight

A symmetric kernel makes the integral operator self-adjoint; compactness (square-integrability) then forces a discrete real spectrum with orthonormal eigenfunctions, and adding continuity plus positivity (Mercer) upgrades the eigenfunction expansion of the kernel from $L^2$ convergence to absolute uniform convergence.

Where this lives today

The kernel that Mercer certified is now the central object of a large slice of machine learning. A support-vector machine or kernel-PCA never touches its feature vectors directly; it works only with the Gram matrix $K_{ij}=K(x_i,x_j)$ , and the entire "kernel trick" is legitimate precisely because Mercer's theorem guarantees that a continuous positive-definite $K$ is the inner product $\langle\Phi(x_i),\Phi(x_j)\rangle$ of a feature map into a genuine Hilbert space, the eigenfunctions $\sqrt{\lambda_n}\phi_n$ being its coordinates. Gaussian-process regression uses the same theorem probabilistically — a covariance function is a Mercer kernel whose Karhunen–Loève modes are the principal directions of function-space uncertainty, with $\lambda_n$ -decay setting the effective degrees of freedom — an idea that reaches deep learning through the neural tangent kernel. On the physics side the trace formula $\int G(x,x)\,\mathrm{d}x=\sum_n\mu_n^{-1}$ feeds spectral-zeta regularization of Casimir energies, and Karhunen–Loève truncation is the workhorse of reduced-order modeling for stochastic PDEs. A live open thread is spectral estimation: recovering an unknown kernel's eigenvalue decay from finitely many samples, which governs both the generalization of kernel machines and the convergence of stochastic-PDE solvers.

The physical picture

Picture the kernel as an infinite continuous matrix $K_{xy}=K(x,y)$ , indexed not by integers but by the points of $[a,b]$ . Hermitian symmetry $K_{xy}=\overline{K_{yx}}$ is reflection across the main diagonal with a conjugate — the condition $A=A^{\dagger}$ a physicist demands of any observable. The spectral theorem then promises a real ladder of eigenvalues and orthogonal eigenfunctions, and Mercer's expansion $K(x,y)=\sum_n\lambda_n\phi_n(x)\overline{\phi_n(y)}$ is just the outer-product decomposition $A=\sum_n\lambda_n\,u_n u_n^{\dagger}$ with a continuous index. The first figure makes this "continuous matrix, diagonalized" the whole picture.

Figure

A Hermitian kernel is a continuous matrix symmetric across its diagonal; the spectral theorem resolves it into real eigenvalues λ_n with orthonormal eigenfunctions φ_n, and Mercer stacks the rank-one pieces λ_n φ_n(x) φ̄_n(y) back into K

Square-integrability makes the operator compact — essentially finite-dimensional, approximable by chopping the eigenladder off after finitely many rungs. That is why the spectrum is discrete with only accumulation point $0$ : the bars in an eigenvalue chart must shrink to nothing, and the decay of $\lambda_n$ measures the smoothness of $K$ . Positivity — Mercer's extra hypothesis, $\langle f,Kf\rangle\ge0$ — recasts the kernel as a Gram matrix of feature vectors $K(x,y)=\langle\Phi(x),\Phi(y)\rangle$ , and it is precisely the ingredient that upgrades the diagonal series from mean-square to uniform convergence. The second figure shows a real spectrum: the Brownian covariance $\min(x,y)$ on $[0,1]$ , whose eigenvalues plunge like $1/n^2$ yet sum to exactly $1/2$ .

Figure

Eigenvalues of the Brownian kernel min(x,y) on the unit interval: λ_n = 1/((n−1/2)²π²) collapse from 0.405 toward zero, while their running total climbs to the Mercer trace Σλ_n = ∫₀¹ x dx = 1/2 exactly

The core equations

An integral operator is built from a kernel as a matrix is built from its entries: the kernel supplies a "matrix element" $K(x,y)$ for every pair of continuous indices, and integration replaces summation over a column.

(K\phi)(x)=\int_a^b K(x,y)\,\phi(y)\,\mathrm{d}y \qquad (1)

where: $K(x,y)$ — the kernel, a function on $[a,b]\times[a,b]$ ; $\phi$ — an element of $L^2(a,b)$ the operator acts on; $K$ — the induced linear operator on $L^2(a,b)$ . Reading: acting with $K$ smears $\phi$ against a $y$ -slice of the kernel to return a function of $x$ . The property that organizes the whole theory is that Hermitian symmetry of the kernel is identical to self-adjointness under the $L^2$ inner product $\langle f,g\rangle=\int_a^b\overline{f}\,g\,\mathrm{d}x$ . Derivation A proves it in one line of Fubini.

K(y,x)=\overline{K(x,y)}\quad\Longleftrightarrow\quad K=K^{\dagger} \qquad (2)

where: $\overline{(\cdot)}$ — complex conjugation, dropped for real kernels where the condition is just $K(x,y)=K(y,x)$ ; $K^{\dagger}$ — the adjoint. Reading: symmetry of the kernel across its diagonal is the continuous-index version of a Hermitian matrix. Self-adjointness dictates the entire structure of the spectrum. As for Hermitian matrices, the eigenvalues are real and eigenfunctions of different eigenvalues are orthogonal, so they can be normalized into an orthonormal system.

K\phi_n=\lambda_n\phi_n,\qquad \lambda_n\in\mathbb{R},\qquad \langle\phi_m,\phi_n\rangle=\delta_{mn} \qquad (3)

where: $\lambda_n$ — the $n$ -th eigenvalue, real by self-adjointness; $\phi_n$ — the $n$ -th eigenfunction, unit $L^2$ norm; $\delta_{mn}$ — the Kronecker delta. Reading: the operator is diagonal in the basis $\{\phi_n\}$ , the $\lambda_n$ on the diagonal. Square-integrability is the analytic engine: it makes $K$ a Hilbert–Schmidt (hence compact) operator, and Parseval in the product space $L^2([a,b]^2)$ ties the kernel's total size to the sum of squared eigenvalues, forcing $\lambda_n\to0$ .

\iint_{[a,b]^2}|K(x,y)|^2\,\mathrm{d}x\,\mathrm{d}y=\sum_n\lambda_n^2<\infty,\qquad \lambda_n\to0 \qquad (4)

where: $\sum_n\lambda_n^2$ — the squared Hilbert–Schmidt (Frobenius) norm; $\lambda_n\to0$ — eigenvalues accumulate only at the origin. Reading: summable squares force a genuinely discrete spectrum. The crown jewel is Mercer's expansion. For a continuous, symmetric, positive semi-definite kernel the bilinear eigenseries does not merely approximate $K$ in mean — it equals $K$ everywhere, absolutely and uniformly, which licenses evaluation on the diagonal. Derivation B is the proof.

K(x,y)=\sum_n\lambda_n\,\phi_n(x)\,\overline{\phi_n(y)}\qquad(\text{absolutely and uniformly}) \qquad (5)

where: $\lambda_n\ge0$ — the nonnegative eigenvalues of a positive semi-definite continuous kernel; $\phi_n$ — the continuous orthonormal eigenfunctions. Reading: the kernel is literally rebuilt, point by point, from its own eigenmodes weighted by their eigenvalues. Setting $y=x$ in (5) and integrating along the diagonal — legal because the convergence is uniform — collapses the double sum into the trace identity, a useful spectral fingerprint.

\operatorname{Tr}K=\sum_n\lambda_n=\int_a^b K(x,x)\,\mathrm{d}x \qquad (6)

where: $\operatorname{Tr}K$ — the trace of the integral operator; $K(x,x)$ — the kernel on its diagonal. Reading: summing the eigenvalues equals integrating the kernel along the diagonal, as the trace of a matrix sums its diagonal entries. Finally, diagonalization is what makes symmetric kernels indispensable in practice. In the eigenbasis a Fredholm equation of the second kind, $(I-\lambda K)\phi=f$ , uncouples into one scalar equation per mode, and the resolvent is a diagonal sum with poles at the characteristic values.

(I-\lambda K)\phi=f\ \Longrightarrow\ \phi=f+\lambda\sum_n\frac{\lambda_n}{1-\lambda\lambda_n}\,\langle\phi_n,f\rangle\,\phi_n \qquad (7)

where: $\lambda$ — the spectral parameter; $\langle\phi_n,f\rangle$ — the $n$ -th coefficient of the data $f$ ; $1/\lambda_n$ — the characteristic value where mode $n$ resonates. Reading: away from the characteristic values the equation inverts mode by mode; at $\lambda=1/\lambda_n$ that mode blows up and the Fredholm alternative takes over.

Derivation AFrom a Hermitian kernel to a real spectrum and orthogonal eigenfunctions

The goal is the algebraic foundation of the whole subject: that symmetry of the kernel is exactly self-adjointness of the operator, and that self-adjointness then forces real eigenvalues and orthogonal eigenfunctions, precisely as it does for a Hermitian matrix. We move the operator across the inner product using Fubini to identify the adjoint kernel, then run the classical eigenvalue sandwich for reality and orthogonality — the only analytic input being that the double integrals may be reordered, which square-integrability guarantees.

Step 1. Write out $\langle f,Kg\rangle=\int_a^b\overline{f(x)}\big(\int_a^b K(x,y)g(y)\,\mathrm{d}y\big)\mathrm{d}x$ and use Fubini to swap the order of integration, conjugating the inner $x$ -integral to expose an $\overline{(\cdot)}\,g$ structure: $\langle f,Kg\rangle=\int_a^b\overline{\big(\int_a^b\overline{K(x,y)}f(x)\,\mathrm{d}x\big)}\,g(y)\,\mathrm{d}y=\langle K^{\dagger}f,g\rangle$ with $(K^{\dagger}f)(y)=\int_a^b\overline{K(x,y)}f(x)\,\mathrm{d}x$ . Why: the adjoint is defined by $\langle f,Kg\rangle=\langle K^{\dagger}f,g\rangle$ , so reading off the kernel of $K^{\dagger}$ reduces self-adjointness to a statement about the kernel alone.

Step 2. Compare kernels: $K^{\dagger}=K$ holds if and only if $\overline{K(x,y)}=K(y,x)$ . So a Hermitian kernel is a self-adjoint operator and conversely — equation (2), with no positivity or continuity assumed. Why: this is the hinge of the article; every spectral conclusion below is borrowed from the theory of self-adjoint operators.

Step 3. Take two eigenpairs $K\phi_m=\lambda_m\phi_m$ , $K\phi_n=\lambda_n\phi_n$ and sandwich: $\langle\phi_m,K\phi_n\rangle=\langle K\phi_m,\phi_n\rangle$ becomes $\lambda_n\langle\phi_m,\phi_n\rangle=\overline{\lambda_m}\langle\phi_m,\phi_n\rangle$ (the left eigenvalue conjugates out of the first slot). Why: this one identity carries both reality and orthogonality, read with $m=n$ and then $m\neq n$ .

Step 4. Set $m=n$ : since $\|\phi_n\|^2>0$ , $\lambda_n=\overline{\lambda_n}$ , so every eigenvalue is real. Set $m\neq n$ : now $(\lambda_n-\lambda_m)\langle\phi_m,\phi_n\rangle=0$ , so distinct eigenvalues force $\langle\phi_m,\phi_n\rangle=0$ ; a degenerate eigenspace is finite-dimensional and orthonormalized by Gram–Schmidt. Why: this reproduces the finite-dimensional spectral theorem for the continuous index, delivering the orthonormal eigenbasis $\{\phi_n\}$ that equations (3)–(7) rely on.

Go deeper · Derivation B · 1 minHow positivity upgrades mean-square convergence to uniform convergence

Derivation BHow positivity upgrades mean-square convergence to uniform convergence

The goal is Mercer's theorem proper: for a continuous, symmetric, positive semi-definite kernel the bilinear eigenseries (5) converges not just in $L^2$ but absolutely and uniformly, so the diagonal value $K(x,x)$ and the trace formula (6) are legitimate. The strategy is a Dini argument — build a monotone increasing sequence of continuous diagonal partial sums bounded above by a continuous limit, make that convergence uniform by Dini's theorem, then bootstrap to the full off-diagonal series with Cauchy–Schwarz. Positivity is used twice and is indispensable.

Step 1. Let $K_N(x,y)=K(x,y)-\sum_{n\le N}\lambda_n\phi_n(x)\overline{\phi_n(y)}$ be the remainder after $N$ modes. Subtracting finitely many positive rank-one pieces $\lambda_n\phi_n\otimes\overline{\phi_n}$ (each $\lambda_n\ge0$ ) from a positive kernel leaves a continuous, symmetric, positive semi-definite kernel. Why: positivity of $K_N$ pins its diagonal below that of $K$ ; without $\lambda_n\ge0$ the argument collapses.

Step 2. A continuous positive semi-definite kernel has a nonnegative diagonal, so $K_N(x,x)=K(x,x)-\sum_{n\le N}\lambda_n|\phi_n(x)|^2\ge0$ . Hence $S_N(x)=\sum_{n\le N}\lambda_n|\phi_n(x)|^2$ increases in $N$ and is bounded above pointwise by $K(x,x)$ . Why: monotone-and-bounded is exactly the hypothesis Dini needs; the diagonal is where positivity buys the upper bound.

Step 3. $S_N(x)\nearrow S(x)\le K(x,x)$ pointwise, with $S$ continuous (it equals $K(x,x)$ ). Dini's theorem — monotone pointwise convergence of continuous functions to a continuous limit on a compact set is uniform — makes $\sum_n\lambda_n|\phi_n(x)|^2$ converge uniformly on $[a,b]$ . Why: this is the one place compactness of the domain is spent; on a non-compact interval Dini can fail and Mercer's theorem is false.

Step 4. Control the off-diagonal tail by Cauchy–Schwarz with the nonnegative weights $\lambda_n$ : $\big|\sum_{n>N}\lambda_n\phi_n(x)\overline{\phi_n(y)}\big|^2\le\big(\sum_{n>N}\lambda_n|\phi_n(x)|^2\big)\big(\sum_{n>N}\lambda_n|\phi_n(y)|^2\big)$ . The right side is a product of diagonal tails, each uniformly small by Step 3, so the full bilinear series converges absolutely and uniformly to $K(x,y)$ ; evaluating at $y=x$ and integrating term by term gives $\int_a^b K(x,x)\,\mathrm{d}x=\sum_n\lambda_n$ . Why: this is the payoff — pointwise legitimacy of the expansion and the trace formula, which bare Hilbert–Schmidt cannot supply.

Go deeper · Derivation C · 1 minThe Green's function of a Sturm–Liouville operator is a Mercer kernel

Derivation CThe Green's function of a Sturm–Liouville operator is a Mercer kernel

The goal is to show that the abstract machinery lands exactly on the eigenfunction expansions physicists already use: the Green's function of a positive self-adjoint Sturm–Liouville operator is a continuous symmetric positive-definite kernel, so Mercer's theorem hands back its bilinear eigenexpansion and a finite trace. Take $L=-\frac{\mathrm{d}}{\mathrm{d}x}\big(p(x)\frac{\mathrm{d}}{\mathrm{d}x}\big)+q(x)$ self-adjoint on $[a,b]$ with separated homogeneous boundary conditions and $L>0$ , eigenpairs $Lu_n=\mu_n u_n$ , $\mu_n>0$ . We identify $G$ as the inverse operator, transfer eigenfunctions, check the Mercer hypotheses, and read off the trace.

Step 1. The Green's function solves $L_x G(x,y)=\delta(x-y)$ with the same boundary conditions; self-adjointness of $L$ makes it symmetric, $G(y,x)=G(x,y)$ , and continuous on $[a,b]^2$ with only a derivative kink across the diagonal. Why: continuity and symmetry are two of Mercer's three hypotheses, both automatic from self-adjointness of a second-order operator.

Step 2. As an integral operator $G=L^{-1}$ , so it shares eigenfunctions with $L$ but inverts the eigenvalues: $Gu_n=\mu_n^{-1}u_n=\lambda_n u_n$ with $\lambda_n=1/\mu_n$ . Why: this converts the differential eigenproblem, whose eigenvalues run to infinity, into an integral one whose eigenvalues collapse to zero — the compact setting Mercer needs.

Step 3. Positivity of $L$ makes every $\mu_n>0$ , hence every $\lambda_n=\mu_n^{-1}>0$ : the third Mercer hypothesis. Weyl's asymptotics give $\mu_n\sim(n\pi/(b-a))^2$ , so $\lambda_n\to0$ and $\sum_n\lambda_n$ converges. Why: positivity plus square-summable eigenvalues promote the $L^2$ expansion to a uniform one.

Step 4. Mercer now yields the uniformly convergent expansion $G(x,y)=\sum_n\mu_n^{-1}u_n(x)u_n(y)$ , whose diagonal gives the trace $\int_a^b G(x,x)\,\mathrm{d}x=\sum_n\mu_n^{-1}=\zeta_L(1)$ , a spectral zeta value. For $-u''$ on $[0,\pi]$ with Dirichlet data, $\mu_n=n^2$ and the trace is $\sum_n n^{-2}=\pi^2/6$ . Why: this is the familiar Green's-function series of physics, now certified pointwise, its trace a finite invariant that reappears as a Casimir energy after regularization.

Limiting cases

The conclusions sharpen, hypothesis by hypothesis, as one climbs from a bare square-integrable kernel to a full Mercer kernel — and weaken again the moment positivity is dropped.

Case	Result	Meaning
Square-integrable only (Hilbert–Schmidt)	$\iint\|K\|^2<\infty\Rightarrow\sum_n\lambda_n^2<\infty$	discrete real spectrum, $\lambda_n\to0$ ; the kernel series converges in $L^2$ , but pointwise values are not controlled
Continuous + positive (Mercer)	$K(x,y)=\sum_n\lambda_n\phi_n(x)\overline{\phi_n(y)}$	absolute, uniform convergence; continuous eigenfunctions; finite trace $\sum_n\lambda_n=\int_a^b K(x,x)\,\mathrm{d}x$
Degenerate / separable kernel	$K(x,y)=\sum_{j=1}^{N}a_j(x)\overline{b_j(y)}$	finite rank: only $N$ nonzero eigenvalues; the Fredholm equation collapses to exact $N\times N$ linear algebra
Indefinite continuous kernel	$\lambda_n$ of both signs	Hilbert–Schmidt still holds, but Mercer fails: the diagonal series need not converge uniformly and $\sum_n\|\lambda_n\|$ may diverge

Common trapMean-square convergence lets you evaluate the eigenseries at a point

It does not. Hilbert–Schmidt gives convergence of $\sum_n\lambda_n\phi_n(x)\overline{\phi_n(y)}$ to $K$ only in $L^2([a,b]^2)$ , in mean square. An $L^2$ function is an equivalence class up to sets of measure zero, and the diagonal $x=y$ is a measure-zero set — so setting $x=y$ to read off $K(x,x)$ is not licensed by mean-square convergence alone. You need Mercer's continuity and positivity to upgrade to uniform convergence before touching the diagonal. Positivity is not decoration: a continuous symmetric kernel with eigenvalues of both signs (say $+1,-1,+\tfrac12,-\tfrac12,\dots$ ) can have a diagonal series that fails to converge uniformly, its formal trace $\sum_n\lambda_n$ not even absolutely convergent while $\sum_n\lambda_n^2$ stays finite.

Worked examples

1 · The Brownian kernel min(x,y) — a fully worked Mercer expansion

The covariance of standard Brownian motion is $K(x,y)=\min(x,y)$ on $[0,1]$ . Find its eigenvalues and eigenfunctions, verify the trace formula, and write the Karhunen–Loève expansion of the process.

Step 1. Recognize the kernel as a Green's function: $\min(x,y)$ equals $x$ for $x<y$ and $y$ for $x>y$ , which is exactly the Green's function of $-\mathrm{d}^2/\mathrm{d}x^2$ with $u(0)=0$ , $u'(1)=0$ . So the eigenproblem $\int_0^1\min(x,y)\phi(y)\,\mathrm{d}y=\lambda\phi(x)$ becomes the ODE $-\phi''=\mu\phi$ with $\mu=1/\lambda$ under those boundary conditions.

Step 2. Solve the ODE. $\phi(x)=\sin(\sqrt{\mu}\,x)$ satisfies $\phi(0)=0$ ; then $\phi'(1)=\sqrt{\mu}\cos\sqrt{\mu}=0$ forces $\sqrt{\mu}=(n-\tfrac12)\pi$ . Hence $\mu_n=(n-\tfrac12)^2\pi^2$ and the operator eigenvalues are $\lambda_n=1/((n-\tfrac12)^2\pi^2)$ , with normalized eigenfunctions $\phi_n(x)=\sqrt2\,\sin\!\big((n-\tfrac12)\pi x\big)$ .

Step 3. Check the numbers. The largest eigenvalue is $\lambda_1=1/((0.5)^2\pi^2)=\mathbf{0.4053}$ , then $\lambda_2=0.0450$ , $\lambda_3=0.0162$ — the plunge plotted in the second figure. The trace formula demands $\sum_n\lambda_n=\int_0^1 x\,\mathrm{d}x=\tfrac12$ ; independently $\sum_n\lambda_n=\frac{1}{\pi^2}\sum_{n\ge1}(n-\tfrac12)^{-2}=\frac{1}{\pi^2}\cdot\frac{\pi^2}{2}=\mathbf{\tfrac12}$ . The two routes agree exactly — a clean confirmation of Mercer's trace identity.

Step 4. Assemble the Karhunen–Loève expansion. With uncorrelated standard-normal $\xi_n$ , $B(x)=\sum_n\sqrt{\lambda_n}\,\xi_n\,\phi_n(x)=\sum_{n\ge1}\frac{\sqrt2\,\xi_n}{(n-\tfrac12)\pi}\sin\!\big((n-\tfrac12)\pi x\big)$ . This is the optimal, variance-minimizing representation of Brownian motion; its coefficients decay only like $1/n$ , reflecting the roughness of the paths. Truncating at $N$ modes keeps a fraction $\sum_{n\le N}\lambda_n/\tfrac12$ of the variance — five modes already retain $0.4798/0.5=96\%$ .

2 · A degenerate kernel and the Fredholm alternative

Solve the Fredholm equation of the second kind $\phi(x)-\mu\int_0^1 xy\,\phi(y)\,\mathrm{d}y=f(x)$ on $[0,1]$ , and find the value of $\mu$ at which solvability fails.

Step 1. The kernel $K(x,y)=xy$ is separable and rank one, so it has a single nonzero eigenvalue. Testing $\phi(x)=x$ in $K\phi=x\int_0^1 y\phi(y)\,\mathrm{d}y$ gives $Kx=x\int_0^1 y^2\,\mathrm{d}y=\tfrac13 x$ , so $\lambda_1=\tfrac13$ with eigenfunction $\phi_1(x)=\sqrt3\,x$ ; every function orthogonal to $x$ is in the null space. The characteristic value is $1/\lambda_1=\mathbf{3}$ .

Step 2. Use the rank-one structure: any solution has the form $\phi(x)=f(x)+\mu x\,c$ with $c=\int_0^1 y\phi(y)\,\mathrm{d}y$ one unknown constant. Substituting back, $c=F+\tfrac{\mu}{3}c$ with $F=\int_0^1 yf(y)\,\mathrm{d}y$ , giving $c=\dfrac{3F}{3-\mu}$ .

Step 3. Read off the solution and its resonance. The full solution $\phi(x)=f(x)+\dfrac{3\mu F}{3-\mu}\,x$ is valid for all $\mu\neq3$ ; at $\mu=3=1/\lambda_1$ the denominator vanishes, the resonance the resolvent (7) predicts. At $\mu=3$ a solution exists only if $F=\int_0^1 yf(y)\,\mathrm{d}y=0$ , i.e. the data is orthogonal to the eigenfunction $x$ — precisely the Fredholm alternative. For $f(x)=x$ , $F=\tfrac13$ and the solution is $\phi(x)=\dfrac{3x}{3-\mu}$ , which diverges as $\mu\to3$ .

Physics of the portrait

Mercer's theorem sits where two currents meet: Hilbert's spectral theory of integral equations, and the classical question of when an eigenfunction series may be summed pointwise. The man who answered it did so in a single 1909 paper, and gave positive-definite kernels their name.

Origins · 1 minJames Mercer and how this was discovered

James Mercer

1883–1932 · integral equations and positive-definite kernels

James Mercer was a British mathematician whose name is attached to one of the load-bearing theorems of functional analysis, though he published relatively little. Born in Bootle, Lancashire, he studied at Manchester and then Trinity College, Cambridge, where he became a Fellow and worked in the orbit of the ideas David Hilbert and Erhard Schmidt were then developing in Göttingen. His decisive contribution was the 1909 memoir "Functions of positive and negative type and their connection with the theory of integral equations", which introduced the kernel of positive type — what we now call positive semi-definite — and proved that a continuous kernel of positive type on a compact interval has an eigenfunction expansion converging absolutely and uniformly. That single theorem separates a merely mean-square eigenexpansion from one you may evaluate at a point, and it is why his name survives in fields he never saw: the Karhunen–Loève expansion, reproducing-kernel Hilbert spaces, and the "Mercer kernels" at the heart of support-vector machines and Gaussian-process learning. Mercer served in the Royal Navy during the First World War; his health declined afterward, and he died in London in 1932 at forty-nine, a Fellow of the Royal Society since 1922.

History

1904

— Hilbert launches the spectral theory of symmetric integral kernels

1907

— Schmidt recasts it through the eigenfunction (Hilbert–Schmidt) expansion

1909

— Mercer proves uniform convergence for continuous positive-definite kernels

1918

— the Fredholm–Riesz theory of compact operators reaches maturity

1947

— Karhunen and Loève build the covariance expansion of random processes on Mercer's theorem

Going further

Texts. Stone & Goldbart, Mathematics for Physics, chapter on integral equations and Hilbert–Schmidt theory · Arfken & Weber, Mathematical Methods for Physicists, chapter on integral equations · Hassani, Mathematical Physics, chapters on compact and Sturm–Liouville operators · Byron & Fuller, Mathematics of Classical and Quantum Physics, chapter on operators in Hilbert space · Courant & Hilbert, Methods of Mathematical Physics, Vol. I, ch. III (eigenfunction theory and Mercer's theorem) · Reed & Simon, Methods of Modern Mathematical Physics I: Functional Analysis (compact self-adjoint operators, spectral theorem) · Riley, Hobson & Bence, Mathematical Methods for Physics and Engineering, integral-equations chapter.

Provenance. Theorem statements follow Stone & Goldbart, Courant & Hilbert Vol. I ch. III, and Reed & Simon Vol. I; the Sturm–Liouville application follows Arfken & Weber and Riley–Hobson–Bence. A verifier should re-derive: (i) self-adjointness ⇔ Hermitian kernel via Fubini; (ii) reality and orthogonality of eigenpairs; (iii) the Parseval identity $\iint|K|^2=\sum_n\lambda_n^2$ ; (iv) the Dini argument for Mercer's uniform convergence; (v) the $\min(x,y)$ example — eigenfunctions $\sqrt2\sin((n-\tfrac12)\pi x)$ , eigenvalues $((n-\tfrac12)\pi)^{-2}$ , trace $\sum_n(n-\tfrac12)^{-2}=\pi^2/2$ against $\int_0^1 x\,\mathrm{d}x=1/2$ ; (vi) the $xy$ kernel with characteristic value $3$ and condition $\int_0^1 yf(y)\,\mathrm{d}y=0$ . Anchor numbers: $\sum 1/n^2=\pi^2/6$ , $\sum(n-\tfrac12)^{-2}=\pi^2/2$ , $\lambda_1(\min)=0.4053$ .

Prerequisites. Hilbert spaces and $L^2$ · Self-adjoint and compact operators · Spectral theorem (finite-dimensional) · Fredholm Integral Equations. Leads to. Green's functions of Sturm–Liouville problems · WKB Approximation · Karhunen–Loève and Reproducing-Kernel Hilbert Spaces.

Ask the tutor: Ask me to walk through the Dini argument for Mercer's theorem, or to work the Karhunen–Loève expansion of any covariance kernel you name.

Why is a symmetric kernel such a big deal — what breaks without symmetry?

Without symmetry the operator is not self-adjoint, and almost every convenience disappears. Eigenvalues can be complex, eigenfunctions need not be orthogonal, and they may not even form a complete set — a non-symmetric compact operator can fail to be diagonalizable, requiring the singular-value decomposition instead. Symmetry is what buys the real spectrum, the orthonormal eigenbasis, and the clean diagonalization in which integral equations decouple mode by mode. It is the continuous-index version of the fact that only Hermitian matrices are guaranteed a real orthonormal eigenbasis — which is why symmetric kernels are the ones physicists can actually solve.

What is the difference between the Hilbert–Schmidt theorem and Mercer's theorem?

They answer two questions about the same series $\sum_n\lambda_n\phi_n(x)\overline{\phi_n(y)}$ . Hilbert–Schmidt needs only square-integrability and gives convergence to $K$ in mean square — an $L^2$ statement that says nothing about any point. Mercer adds continuity on a compact square and positive semi-definiteness, and concludes absolute, uniform convergence, hence pointwise everywhere. The upshot is the diagonal: only Mercer lets you set $x=y$ , write $K(x,x)=\sum_n\lambda_n|\phi_n(x)|^2$ , and integrate to the trace formula. Hilbert–Schmidt gives existence of the expansion; Mercer gives its pointwise legitimacy.

Why must the eigenvalues accumulate only at zero?

Because a Hilbert–Schmidt kernel makes $K$ compact, and compactness forces the eigenvalues to pile up only at the origin. Square-integrability gives $\sum_n\lambda_n^2<\infty$ , so only finitely many eigenvalues can exceed any fixed threshold — if infinitely many stayed bounded away from zero, their squares could not sum. Physically, a smooth kernel cannot resolve arbitrarily fine high-frequency eigenfunctions, so it assigns them vanishingly small eigenvalues. This is also why an integral operator, unlike its inverse differential operator, has a spectrum that collapses toward zero rather than running to infinity — the Green's-function derivation makes the inverse relation explicit.

Does Mercer's theorem require the kernel to be strictly positive, or is semi-definite enough?

Positive semi-definite is enough: the hypothesis is $\langle f,Kf\rangle\ge0$ for all $f$ , equivalently every eigenvalue $\lambda_n\ge0$ , and zero eigenvalues are allowed. Eigenfunctions with $\lambda_n=0$ simply drop out of the expansion, so the sum runs effectively over the strictly positive eigenvalues. What is not allowed is a negative eigenvalue: a single one breaks the Dini monotonicity argument, because the diagonal partial sums stop increasing, and uniform convergence can genuinely fail. So "positive type" in Mercer's original phrase means "no negative eigenvalues", not "no zero eigenvalues".

How does the smoothness of the kernel show up in its eigenvalues?

As the decay rate. A merely continuous kernel has $\lambda_n\to0$ with $\sum_n\lambda_n^2<\infty$ ; adding derivatives speeds the decay, a $C^k$ kernel typically giving $\lambda_n\sim n^{-(k+1)}$ and an analytic kernel exponentially. A smooth kernel averages out fine detail, barely responding to highly oscillatory eigenfunctions and assigning them tiny eigenvalues. This is why the Brownian kernel $\min(x,y)$ , only Lipschitz (its diagonal derivative jumps), has the slow decay $\lambda_n\sim n^{-2}$ matching the roughness of Brownian paths — and why smoother covariance kernels give Gaussian processes with smoother sample functions.

Where does the Karhunen–Loève expansion come from, and why is it "optimal"?

It is Mercer's theorem applied to a covariance kernel. A random field's covariance $C(x,y)=\mathbb{E}[X(x)X(y)]$ is symmetric and positive semi-definite (since $\langle f,Cf\rangle=\mathbb{E}[|\int f X|^2]\ge0$ ); if continuous, Mercer gives $C=\sum_n\lambda_n\phi_n\otimes\phi_n$ . Expanding the field in the same eigenbasis, $X(x)=\sum_n\sqrt{\lambda_n}\,\xi_n\phi_n(x)$ , yields uncorrelated unit-variance coefficients $\xi_n$ . It is optimal in mean square: truncating at the $N$ largest eigenvalues captures the most variance of any $N$ -term linear expansion, because the eigenfunctions are the principal axes of the covariance — PCA lifted from vectors to functions.