Infinite-Dimensional Function Spaces

Function-space geometry is quietly at work far beyond the textbook. Every quantum computation is bookkeeping in a Hilbert space, and the mathematical foundations of quantum field theory still turn on making infinite-dimensional operator algebras and their spectra precise — the rigged Hilbert spaces that hold scattering states with continuous spectra are a direct descendant of the unbounded-operator subtleties above. In data science, kernel methods and support-vector machines lift finite data into a reproducing-kernel Hilbert space, where the "kernel trick" is exactly the integral operator of equation (7) reused as a similarity measure; Gaussian-process regression is a probability distribution over a function space. Signal processing runs on truncated orthonormal expansions: the discrete cosine transform behind JPEG and MPEG is Worked Example 2 in commercial form, keeping the low-order coefficients that carry most of the energy and discarding the rest, while compressed sensing exploits sparsity in a chosen basis to reconstruct MRI images from far fewer samples than classical sampling would demand. Spectral methods integrate the equations of weather and climate by projecting the atmosphere onto spherical-harmonic bases, and finite-element solvers do the same with local bases. And the subject still has open frontiers: the invariant subspace problem — whether every bounded operator on an infinite-dimensional separable Hilbert space has a nontrivial closed invariant subspace — remains unsolved, a reminder that the geometry Hilbert glimpsed is not yet fully mapped.

Picture a string clamped at both ends, plucked and released. At any instant its shape is a function $f(x)$, and the normal modes $\sin(n\pi x/L)$ are a set of mutually perpendicular directions in an abstract space. Giving the amplitude of each mode is exactly like giving the components of a vector along orthogonal axes — except that here the axes are infinite in number. The inner product $\langle e_n,f\rangle=\int e_n^*f\,\mathrm{d}x$ measures the overlap of $f$ with the $n$-th axis, so the Fourier coefficient $c_n$ is a projection, and reconstructing $f=\sum_n c_n e_n$ is reassembling the vector from its shadows. The first figure makes the geometry literal: one plucked profile, resolved into its lowest modes.

Two features of the picture are new to infinite dimension, and both are decisive. First, completeness: the space must have no holes. A sequence of continuous ramps can converge in $L^2$ to a discontinuous step, and unless that step is already a member of the space, the "limit" has fallen out — just as $\sqrt2$ falls out of $\mathbb{Q}$. Completing the continuous functions plugs every such hole and yields $L^2$. Second, the balance sheet of energy must close: a complete orthonormal basis loses nothing, so the norm of $f$ equals the total of its squared coordinates, $\|f\|^2=\sum_n|c_n|^2$. The second figure is that ledger in action — how fast the squared coordinates of a square wave accumulate to the full norm as harmonics are added.

Everything starts with the observation that functions can be added and scaled, so they inherit the algebra of vectors before any analysis is added.

where: $f,g$ — functions regarded as vectors on a common domain; $\alpha$ — scalar from the field $\mathbb{R}$ or $\mathbb{C}$; $x$ — a point of the domain $[a,b]$. Every vector-space axiom holds because it holds for the numbers $f(x),g(x)\in\mathbb{C}$ at each individual $x$; the zero vector is the function $0(x)=0$, and the inverse of $f$ is $-f$. Geometry enters through an inner product, which turns the algebraic space into one with length and angle. On a weighted $L^2$ space it is an integral rather than a finite sum.

where: $f^*$ — complex conjugate of $f$; $w(x)\ge 0$ — weight function, nonzero almost everywhere and often simply $w\equiv1$; $[a,b]$ — interval of definition, possibly unbounded. It is conjugate-symmetric ($\langle g,f\rangle=\langle f,g\rangle^*$), linear in the second slot (physics convention), and positive-definite, and the last property forces us to identify functions equal almost everywhere. The inner product supplies a norm — the length of a function-vector — and the finiteness of that length is precisely the membership card for $L^2$.

where: $\|f\|$ — length of the function-vector; $|f|^2$ — modulus squared, read physically as a local energy or probability density. A function belongs to $L^2[a,b]$ exactly when this integral converges; membership is a global condition, indifferent to the value of $f$ at any single point. Length and inner product are tied together by the single most-used inequality in the subject, valid in every inner-product space regardless of dimension.

where: $\langle f,g\rangle$ — inner product, or overlap integral, of the two functions; $\|f\|,\|g\|$ — their norms. Equality holds only when $f$ and $g$ are parallel; the inequality is what makes the triangle inequality $\|f+g\|\le\|f\|+\|g\|$ — and hence the very notion of distance — work in $L^2$. The heart of the theory is that a complete orthonormal set behaves like an axis system: every function is the sum of its projections, with the coefficients read off by inner products.

where: $\{e_n\}$ — complete orthonormal set (Fourier modes, Legendre, Hermite, or Bessel functions); $\delta_{mn}$ — Kronecker delta; $c_n$ — generalized Fourier coefficient, the projection of $f$ onto $e_n$. The word complete is doing heavy lifting: it guarantees the series actually reconstructs $f$ and misses no part of it. Because no part is missed, the squared length of $f$ equals the total of its squared coordinates — the Pythagorean theorem, promoted to infinitely many axes.

where: $a_n=\langle e_n,f\rangle$, $b_n=\langle e_n,g\rangle$ — coordinate sequences of $f$ and $g$. This is Parseval's (or Plancherel's) identity, and it is the exact statement that the coordinate map $f\mapsto(c_n)$ is an isometry from $L^2$ onto the sequence space $\ell^2$. Operators are the next layer: they generalize matrices, and the continuous analogue of a matrix acting on a vector is an integral operator with a kernel.

where: $K(x,y)$ — kernel, the continuous analogue of a matrix entry $A_{ij}$; $\hat K$ — the linear operator sending $f\mapsto\hat K f$. Summation over the column index $j$ becomes integration over $y$, and the row index $i$ becomes the output point $x$. The operators that matter physically are symmetric under the inner product, which in infinite dimension is a statement about boundary terms, not just about the formula for the operator.

where: $\hat A^\dagger$ — the adjoint of $\hat A$; the equivalence defines (formal) self-adjointness. For a differential operator the identity holds only once boundary conditions kill a surface term, so the domain of admissible functions is part of the operator, not an afterthought. The archetype that manufactures orthonormal bases across physics is the Sturm–Liouville operator, whose eigenfunctions are the sines, Legendre polynomials, and Bessel functions of separation of variables.

where: $p,q,w$ — coefficient functions with $p,w>0$ on $(a,b)$; $\lambda$ — eigenvalue, discrete for a regular problem; $u$ — eigenfunction, orthonormal in $L^2_w$. Self-adjointness of $\hat L$ is what guarantees the eigenvalues are real and the eigenfunctions form a complete orthonormal basis.

Function spaces are stratified by how much structure they carry, and each level recovers a more familiar object as a limiting case.

1 · Parseval's identity and the Basel sum (fully worked)

Expand $f(x)=x$ on $[0,\pi]$ in the orthonormal sine basis $e_n(x)=\sqrt{2/\pi}\,\sin(nx)$, then apply Parseval's identity to evaluate $\sum_{n=1}^\infty 1/n^2$ in closed form.

Step 1. Compute the coefficients. With $\int_0^\pi x\sin(nx)\,\mathrm{d}x=[-x\cos(nx)/n+\sin(nx)/n^2]_0^\pi=\pi(-1)^{n+1}/n$, the projection is $c_n=\langle e_n,f\rangle=\sqrt{2/\pi}\cdot\pi(-1)^{n+1}/n=\sqrt{2\pi}\,(-1)^{n+1}/n$, so $|c_n|^2=2\pi/n^2$. The set $\{e_n\}$ is a complete orthonormal basis of $L^2[0,\pi]$ (the Dirichlet Laplacian eigenfunctions), so Parseval applies.

Step 2. Compute the norm directly: $\|f\|^2=\int_0^\pi x^2\,\mathrm{d}x=\pi^3/3$, so $\|f\|^2=\pi^3/3$.

Step 3. Impose Parseval, $\|f\|^2=\sum_n|c_n|^2$: $\dfrac{\pi^3}{3}=\sum_{n=1}^\infty\dfrac{2\pi}{n^2}=2\pi\sum_{n=1}^\infty\dfrac1{n^2}$. Solving gives $\displaystyle\sum_{n=1}^\infty\frac1{n^2}=\frac{\pi^3/3}{2\pi}=\frac{\pi^2}{6}$, the Basel sum. $\sum 1/n^2=\pi^2/6\approx1.6449$.

Step 4. Sanity-check the convergence. The partial sums $1,\ 1.2500,\ 1.3611,\ 1.4236,\ 1.4636,\dots$ climb steadily toward $\pi^2/6=1.64493$. The value is not an approximation forced by truncation; it is the exact statement that a plucked-string profile stores its entire mean-square in its harmonic amplitudes — the Pythagorean theorem, counted over infinitely many axes.

2 · How many harmonics hold the energy of a square wave

The square wave $f(x)=+1$ on $(0,\pi)$ and $-1$ on $(-\pi,0)$ is expanded on $[-\pi,\pi]$. What fraction of its total energy $\|f\|^2$ is captured by the first few harmonics, and what does the answer say about best approximation?

Step 1. Set up coefficients in the orthonormal set $e_n=\sin(nx)/\sqrt\pi$. Because $f$ is odd, every cosine and constant coefficient vanishes, and $c_n=\tfrac1{\sqrt\pi}\int_{-\pi}^{\pi}f\sin(nx)\,\mathrm{d}x=\tfrac1{\sqrt\pi}\cdot\tfrac{2(1-(-1)^n)}{n}$. This is zero for even $n$ and $4/(n\sqrt\pi)$ for odd $n$, so $|c_n|^2=16/(\pi n^2)$ for odd $n$.

Step 2. Get the total energy: $\|f\|^2=\int_{-\pi}^{\pi}1\,\mathrm{d}x=2\pi$. The captured fraction after odd harmonics up to $N$ is $\dfrac1{2\pi}\sum_{\text{odd }n\le N}\dfrac{16}{\pi n^2}=\dfrac{8}{\pi^2}\sum_{\text{odd }n\le N}\dfrac1{n^2}$, with prefactor $8/\pi^2=0.81057$.

Step 3. Evaluate the running total. The fundamental alone gives $8/\pi^2=\mathbf{81.1\%}$; adding $n=3$ raises it to $0.81057\times(1+\tfrac19)=\mathbf{90.1\%}$; through $n=5$, $\mathbf{93.3\%}$; through $n=7$, $95.0\%$; through $n=9$, $96.0\%$. As a check, the full sum uses $\sum_{\text{odd}}1/n^2=\pi^2/8$, which returns exactly $\|f\|^2=2\pi$ — Parseval closes.

Step 4. Interpret through best approximation. By Derivation 1 the $N$-term truncation is the closest element of its subspace to $f$, so these percentages are the best possible for that many modes — no other choice of coefficients beats the Fourier coefficients. That a single mode already holds four-fifths of the energy is exactly why truncated orthonormal expansions power lossy compression and spectral solvers.

The theory of function spaces is one of the rare parts of physics whose central object carries a person's name. "Hilbert space" honors the mathematician who, studying integral equations at Göttingen, first found the geometry of infinitely many coordinates hiding inside analysis — though, characteristically, it was others who gave it its final axioms.

Texts. Stone & Goldbart, Mathematics for Physics, Ch. 1–2 (function spaces, operators, Hilbert space) · Arfken, Weber & Harris, Mathematical Methods for Physicists, 7th ed., Ch. 5 and 8 · Hassani, Mathematical Physics, Ch. 5–6 (Hilbert spaces and operators) · Byron & Fuller, Mathematics of Classical and Quantum Physics, Ch. 3–5 · Riley, Hobson & Bence, Mathematical Methods for Physics and Engineering, 3rd ed., Ch. 17 · Kreyszig, Introductory Functional Analysis with Applications, Ch. 2–3 · Reed & Simon, Methods of Modern Mathematical Physics I: Functional Analysis. Key identities in this article: the completing-the-square identity of Derivation 1 yields Bessel's inequality as a finite-$N$ reduction to orthogonal projection; the integral $\int_0^{1/n}(nx-1)^2\,\mathrm{d}x=1/(3n)$ of Derivation 2 pins the convergence rate exactly; Derivation 3 uses the Lagrange identity to establish orthogonality. Worked Example 1 gives $\sum1/n^2=\pi^2/6\approx1.6449$ from Parseval with $\|f\|^2=\pi^3/3$ and $|c_n|^2=2\pi/n^2$; Worked Example 2 gives energy fractions $8/\pi^2\approx81.1\%$, $90.1\%$, $93.3\%$ with total $\|f\|^2=2\pi$ via $\sum_{\text{odd}}1/n^2=\pi^2/8$. Every relation here is a mathematical identity; no empirical constants appear.

Builds on: Vector Spaces and Subspaces · Linear Independence, Basis, and Dimension · Inner-Product Spaces and Norms · Orthogonality and the Gram–Schmidt Process. Leads to: Adjoint and Hermitian Operators · Diagonalization and the Spectral Decomposition · Linear Operators and Matrix Representations · Eigenvalues, Eigenvectors, and the Characteristic Equation.

Ask the tutor: Ask me to expand any function you like in an orthonormal basis and watch Parseval's identity balance the energy books term by term.