Hilbert Spaces

A Hilbert space is a complete inner product space — a vector space equipped with an inner product whose induced metric makes every Cauchy sequence converge. It is the natural setting where geometry (lengths, angles, orthogonality, projection) survives into infinite dimensions, and where Fourier-style expansions converge. Used in quantum mechanics, signal processing, PDEs, and probability. Named for David Hilbert; the abstract axioms were formalized by John von Neumann (1929–1932).

A Hilbert space $H$ is a real or complex vector space carrying an inner product $⟨ \cdot, \cdot ⟩$ , which induces a norm $∥ x ∥ = ⟨ x, x ⟩$ and a metric $d (x, y) = ∥ x - y ∥$ ; the defining extra requirement is completeness: every Cauchy sequence converges in $H$ . This combination matters because the inner product gives a notion of orthogonality and projection that lets infinite-dimensional problems be solved by the same geometric reasoning used in $R^{n}$ . The most surprising fact is that every separable infinite-dimensional Hilbert space is isometrically isomorphic to one space, $ℓ^{2}$ — so up to isomorphism there is essentially one such space, no matter whether its elements are sequences, functions, or quantum states. Hilbert spaces are the rigorous home of quantum mechanics and the convergence theory of Fourier series.

The Core Insight

An inner product imports the full geometry of Euclidean space — orthogonality, the Pythagorean theorem, and orthogonal projection onto closed subspaces — into infinite dimensions. Completeness is the glue that makes infinite expansions $x = \sum c_{n} e_{n}$ actually converge to a point that exists. Together they guarantee a "best approximation" always exists, which is why Hilbert spaces dominate analysis.

Formal Statement

Inner Product Space

Let $H$ be a vector space over a field $F$ (either $R$ or $C$ ). An inner product is a map $⟨ \cdot, \cdot ⟩ : H \times H \to F$ satisfying, for all $x, y, z \in H$ and $α \in F$ :

⟨ x, x ⟩ \geq 0, ⟨ x, x ⟩ = 0 ⟺ x = 0

⟳positive-definiteness — this is what makes

∥ x ∥ = ⟨ x, x ⟩

a genuine norm rather than a seminorm.

⟨ α x + z, y ⟩ = α ⟨ x, y ⟩ + ⟨ z, y ⟩

⟳linearity in the first argument; combined with [3] this gives conjugate-bilinearity (the physics convention puts the conjugate-linear slot first).

⟨ x, y ⟩ = \overline{⟨ y, x ⟩}

⟳conjugate symmetry; over

R

the bar vanishes and the inner product is symmetric. Setting

y = x

forces

⟨ x, x ⟩ \in R

, making [1] meaningful.

Here $x, y, z$ are vectors, $α$ a scalar, and $\overline{\cdot}$ denotes complex conjugation. The induced norm is $∥ x ∥ = ⟨ x, x ⟩$ .

Hilbert Space

A Hilbert space is an inner product space $(H, ⟨ \cdot, \cdot ⟩)$ that is complete with respect to the metric $d (x, y) = ∥ x - y ∥$ — that is, every Cauchy sequence $(x_{n})$ in $H$ converges to a limit $x \in H$ :

(\forall ε > 0 \exists N \forall m, n \geq N : ∥ x_{m} - x_{n} ∥ < ε) ⟹ \exists x \in H : ∥ x_{n} - x ∥ \to 0

⟳completeness is a topological condition on the metric, not an algebraic one; an incomplete inner product space is called a pre-Hilbert space and its completion is a Hilbert space.

Cauchy–Schwarz Inequality

For all $x, y$ in an inner product space,

∣ ⟨ x, y ⟩ ∣ \leq ∥ x ∥ ∥ y ∥,

with equality if and only if

x

and

y

are linearly dependent.

⟳for

λ = ⟨ x, y ⟩ /∥ y ∥^{2}

, expand

0 \leq ∥ x - λ y ∥^{2}

and simplify.

Parallelogram Law

A normed space is an inner product space iff its norm satisfies

∥ x + y ∥^{2} + ∥ x - y ∥^{2} = 2∥ x ∥^{2} + 2∥ y ∥^{2} .

⟳expand both sides using

∥ u ∥^{2} = ⟨ u, u ⟩

; the cross terms cancel. The converse (Jordan–von Neumann) recovers

⟨ \cdot, \cdot ⟩

via the polarization identity.

Riesz Representation Theorem

Every continuous linear functional $f : H \to F$ on a Hilbert space is represented by a unique $y_{f} \in H$ :

f (x) = ⟨ x, y_{f} ⟩ for all x \in H, ∥ f ∥ = ∥ y_{f} ∥.

⟳if

f \equiv 0

take

y_{f} = 0

; otherwise project onto

(ker f)^{⊥}

, which is one-dimensional, and scale.

Why It Holds / Derivation

Cauchy–Schwarz via the Projection Trick

If $y = 0$ both sides are $0$ ; assume $y \neq = 0$ .
For any scalar $λ$ , positivity [1] gives

0 \leq ∥ x - λ y ∥^{2} = ∥ x ∥^{2} - \overline{λ} ⟨ x, y ⟩ - λ \overline{⟨ x, y ⟩} + ∣ λ ∣^{2} ∥ y ∥^{2} .

⟳expand

⟨ x - λ y, x - λ y ⟩

using linearity [2] and conjugate symmetry [3].

Choose $λ = ⟨ x, y ⟩ /∥ y ∥^{2}$ to minimize the right side (this is the projection of $x$ onto $y$ ).
Substituting yields

0 \leq ∥ x ∥^{2} - \frac{∣ ⟨ x , y ⟩ ∣ ^{2}}{∥ y ∥ ^{2}},

⟳the two cross terms combine to

- 2∣ ⟨ x, y ⟩ ∣^{2} /∥ y ∥^{2}

and the last term contributes

+ ∣ ⟨ x, y ⟩ ∣^{2} /∥ y ∥^{2}

Rearranging gives $∣ ⟨ x, y ⟩ ∣^{2} \leq ∥ x ∥^{2} ∥ y ∥^{2}$ ; take square roots. Equality forces $x - λ y = 0$ , i.e. linear dependence. $■$

Hilbert Projection Theorem (existence of best approximation)

Let $C \subseteq H$ be nonempty, closed, and convex, and $x \in H$ . We show a unique nearest point $p \in C$ exists.

Let $d = in f_{c \in C} ∥ x - c ∥$ and pick $c_{n} \in C$ with $∥ x - c_{n} ∥^{2} \to d^{2}$ .
Apply the parallelogram law [6] to $u = x - c_{m}$ and $v = x - c_{n}$ :

∥ c_{m} - c_{n} ∥^{2} = 2∥ x - c_{m} ∥^{2} + 2∥ x - c_{n} ∥^{2} - 4 x - \frac{c _{m} + c _{n}}{2}^{2} .

⟳this is [6] rearranged; the midpoint

\frac{c _{m} + c _{n}}{2} \in C

by convexity, so its distance from

x

\geq d

Hence $∥ c_{m} - c_{n} ∥^{2} \leq 2∥ x - c_{m} ∥^{2} + 2∥ x - c_{n} ∥^{2} - 4 d^{2} \to 0$ , so $(c_{n})$ is Cauchy.
By completeness [4], $c_{n} \to p$ , and since $C$ is closed, $p \in C$ with $∥ x - p ∥ = d$ .
Uniqueness: if $p, p^{'}$ both attain $d$ , step 2 with $c_{m} = p, c_{n} = p^{'}$ forces $∥ p - p^{'} ∥ = 0$ . $■$

This is precisely where completeness is indispensable: without it the minimizing sequence need not converge inside the space.

John von Neumann | 1903–1957

Von Neumann gave Hilbert spaces their modern axiomatic definition in 1929 and used them to put quantum mechanics on rigorous mathematical footing in his 1932 Mathematical Foundations of Quantum Mechanics. He unified Heisenberg's matrix mechanics and Schrödinger's wave mechanics by showing both are representations of operators on the same abstract separable Hilbert space — the matrices act on $ℓ^{2}$ , the wavefunctions live in $L^{2}$ , and these spaces are isomorphic. His spectral theorem for unbounded self-adjoint operators is the backbone of observables and measurement in quantum theory.

Building Intuition

Think of a Hilbert space as $R^{n}$ with infinitely many axes that you are allowed to use — provided your coordinates are "square-summable" so the vector has finite length. The inner product measures how much two vectors point in the same direction; orthogonality ( $⟨ x, y ⟩ = 0$ ) means they share no direction at all. Any vector can be split into a part inside a closed subspace and a part perpendicular to it (orthogonal projection), exactly as a shadow on a wall plus the perpendicular gap. Completeness is the promise that if you keep adding more and more orthogonal pieces and the tail energy shrinks, the infinite sum genuinely lands on a vector that exists in the space — no points are "missing." This is why Fourier series of a function converge to something, and Hilbert space tells you that something is the original function.

Orthogonal projection of x onto a closed subspace M: the nearest point p and the perpendicular residual x−p

Worked Examples

Best approximation in $\ell^2$

Given: In

ℓ^{2}

, let

x = (1, \frac{1}{2}, \frac{1}{3}, \frac{1}{4}, \dots)

(entries

1/ n

, which is square-summable since

\sum 1/ n^{2} = π^{2} /6

). Let

M = \overline{span} {e_{1}, e_{2}}

, the closed subspace of sequences supported on the first two coordinates. Find the orthogonal projection

p

and the residual norm. Steps:

Projection onto coordinate axes just keeps those coordinates:

p = ⟨ x, e_{1} ⟩ e_{1} + ⟨ x, e_{2} ⟩ e_{2} = 1 \cdot e_{1} + \frac{1}{2} \cdot e_{2} = (1, \frac{1}{2}, 0, 0, \dots) .

⟳

{e_{1}, e_{2}}

is orthonormal, so

proj_{M} x = \sum_{k} ⟨ x, e_{k} ⟩ e_{k}

Residual $x - p = (0, 0, \frac{1}{3}, \frac{1}{4}, \dots)$ .
Residual energy:

∥ x - p ∥^{2} = n = 3 \sum \infty \frac{1}{n ^{2}} = \frac{π ^{2}}{6} - 1 - \frac{1}{4} = \frac{π ^{2}}{6} - \frac{5}{4} .

⟳total energy

\sum 1/ n^{2} = π^{2} /6

minus the captured terms

1^{2} + (1/2)^{2}

Answer:

p = (1, \frac{1}{2}, 0, 0, \dots), ∥ x - p ∥ = \frac{π ^{2}}{6} - \frac{5}{4} \approx 0.3434

(dimensionless).

:::example[Fourier coefficients in $L^{2} [- π, π]$ ] Given: In the real Hilbert space $L^{2} [- π, π]$ with $⟨ f, g ⟩ = \int_{- π}^{π} f g d t$ , expand $f (t) = t$ in the orthogonal system ${sin (n t)}_{n \geq 1}$ . Compute the coefficient $c_{n}$ and verify Parseval numerically through $n = 3$ .

Steps:

Norm of basis function: $∥ sin (n t) ∥^{2} = \int_{- π}^{π} sin^{2} (n t) d t = π$ .
Coefficient (the orthonormal version uses $sin (n t) / π$ ):

c_{n} = \frac{⟨ t , sin n t ⟩}{∥ sin n t ∥ ^{2}} = \frac{1}{π} \int_{- π}^{π} t sin (n t) d t = \frac{2 ( - 1 ) ^{n + 1}}{n} .

⟳integrate by parts; the

cos

boundary term gives

- 2 π cos (nπ) / n = 2 π (- 1)^{n + 1} / n

, then divide by

π

Parseval: $∥ f ∥^{2} = \int_{- π}^{π} t^{2} d t = \frac{2 π ^{3}}{3} \approx 20.67$ . The expansion gives $\sum_{n} c_{n}^{2} π = π \sum_{n} \frac{4}{n ^{2}}$ .

π n = 1 \sum \infty \frac{4}{n ^{2}} = 4 π \cdot \frac{π ^{2}}{6} = \frac{2 π ^{3}}{3} .

⟳

\sum 1/ n^{2} = π^{2} /6

; the partial sum through

n = 3

gives

4 π (1 + \frac{1}{4} + \frac{1}{9}) = 4 π (1.361) \approx 17.10

, climbing toward

20.67

Answer:

c_{n} = \frac{2 ( - 1 ) ^{n + 1}}{n}

; Parseval's identity confirms

∥ f ∥^{2} = \frac{2 π ^{3}}{3}

, with partial sums converging to it.

Common Misconceptions

Completeness is not optional, and not every inner product space has it. The space

C [0, 1]

of continuous functions with the

L^{2}

inner product is an inner product space but is not a Hilbert space — Cauchy sequences of continuous functions can converge (in

L^{2}

) to discontinuous limits outside the space. Its completion is

L^{2} [0, 1]

, which is a Hilbert space. Also beware: a "basis" of a Hilbert space (an orthonormal basis / Schauder basis) is generally not a Hamel (algebraic) basis — infinite-dimensional Hilbert spaces have uncountable Hamel bases, but separable ones have countable orthonormal bases where convergence is in norm, not finite sums.

Not every norm comes from an inner product

Consider $R^{2}$ with the sup norm $∥ (a, b) ∥_{\infty} = max (∣ a ∣, ∣ b ∣)$ . Take $x = (1, 0)$ , $y = (0, 1)$ . Then $∥ x + y ∥_{\infty}^{2} + ∥ x - y ∥_{\infty}^{2} = 1 + 1 = 2$ , but $2∥ x ∥^{2} + 2∥ y ∥^{2} = 2 + 2 = 4$ . The parallelogram law [6] fails, so no inner product induces this norm — $(R^{2}, ∥ \cdot ∥_{\infty})$ is a Banach space but not a Hilbert space. The same holds for every $ℓ^{p}$ and $L^{p}$ with $p \neq = 2$ : only $p = 2$ is Hilbert.

Connections

Hilbert spaces are the geometrically richest members of the family of Banach Spaces (complete normed spaces); the extra inner-product structure adds orthogonality and self-duality (via Riesz Representation Theorem) that general Banach spaces lack. They are the natural domain of Bounded Linear Operators and the setting for the Spectral Theorem, which diagonalizes self-adjoint operators just as symmetric matrices are diagonalized in Linear Algebra. Convergence of Fourier Series is best understood as orthonormal expansion in $L^{2}$ . The completion construction relates them to Metric Space Completion, and the prototype $ℓ^{2}$ links them to Sequence Spaces.

Real-World Applications

Quantum mechanics: States are unit vectors in a Hilbert space, observables are self-adjoint operators, and measurement probabilities are squared inner products $∣ ⟨ ψ, ϕ ⟩ ∣^{2}$ (Born rule). The standard space is $L^{2} (R^{3})$ for a particle.
Signal processing & JPEG/MP3 compression: Signals are vectors in $L^{2}$ ; expansion in orthonormal wavelet or cosine bases (DCT) lets one discard small coefficients with minimal $L^{2}$ error — exactly best-approximation by projection.
Least-squares regression & data fitting: Fitting a model is orthogonal projection of the data vector onto the column space of the design matrix; the normal equations are Riesz Representation Theorem in finite dimensions.
Reproducing Kernel Hilbert Spaces (RKHS): The mathematical core of support vector machines and kernel methods in machine learning — kernels define inner products in high-dimensional feature spaces.
Partial differential equations: Weak/variational formulations (Sobolev spaces $H^{1}$ , $H^{2}$ ) solve elliptic PDEs via the Lax–Milgram theorem, a coercive cousin of Riesz representation.
Quantum information & qubits: Finite-dimensional $C^{2^{n}}$ Hilbert spaces describe $n$ -qubit systems; entanglement is non-factorizability of tensor-product Hilbert spaces.
Probability theory: Square-integrable random variables form $L^{2} (Ω, F, P)$ ; conditional expectation is orthogonal projection onto a sub- $σ$ -algebra, and uncorrelated means orthogonal.

Historical Development

1900–1910: David Hilbert studies integral equations and the space of square-summable sequences $ℓ^{2}$ , introducing infinitely many variables; the geometry is implicit, not yet axiomatized.
1907: Frigyes Riesz and Ernst Fischer prove the Riesz–Fischer theorem, establishing that $L^{2}$ is complete and isomorphic to $ℓ^{2}$ — the first deep "uniqueness" result.
1907: Riesz proves the representation theorem for continuous linear functionals on $L^{2}$ .
1918: Erhard Schmidt and Riesz develop the geometry of $ℓ^{2}$ — orthogonality, projection — making the Euclidean analogy explicit.
1922: Stefan Banach axiomatizes complete normed spaces (Banach spaces) in his thesis, providing the broader framework.
1929: John von Neumann gives the first abstract axiomatic definition of "Hilbert space" (complete, separable, infinite-dimensional inner product space) in his work on quantum mechanics.
1932: Von Neumann publishes Mathematische Grundlagen der Quantenmechanik, proving the spectral theorem for unbounded self-adjoint operators and unifying matrix and wave mechanics.
1934: Jordan and von Neumann prove the parallelogram law characterization — exactly which norms come from inner products.
1930s–40s: The separability requirement is dropped; general (possibly non-separable) Hilbert spaces become standard, and the theory of operator algebras (von Neumann algebras) grows out of it.

Subtleties & Limits

The invariant subspace problem remains open for separable Hilbert spaces: does every bounded linear operator on an infinite-dimensional separable Hilbert space have a nontrivial closed invariant subspace? Per Enflo (1976/1987) and Charles Read constructed counterexamples on certain Banach spaces, but the Hilbert space case is unresolved — a striking reminder that even in the most structured infinite-dimensional setting, operator theory hides deep mysteries. Related subtleties: in infinite dimensions the closed unit ball is not compact (Riesz's lemma), so existence arguments must use weak compactness or completeness rather than the finite-dimensional Heine–Borel intuition; and bounded operators need not attain their norm.