Functional analysis — Hilbert & Banach spaces

Functional analysis studies infinite-dimensional vector spaces equipped with a topology compatible with their linear structure, focusing on Banach spaces (complete normed spaces) and Hilbert spaces (complete inner product spaces). Key result: the geometry of Hilbert spaces (orthogonality, projection, self-duality) generalizes Euclidean geometry to infinite dimensions, while Banach space theory captures the broader world where norms need not come from inner products. Applies to PDE, quantum mechanics, signal processing, ergodic theory, and numerical analysis. Founded by Hilbert, Fréchet, Riesz, and Banach (1900s–1930s).

A Banach space is a vector space $X$ over $R$ or $C$ equipped with a norm $∥ \cdot ∥$ under which $X$ is complete (every Cauchy sequence converges). A Hilbert space $H$ is a Banach space whose norm arises from an inner product, $∥ x ∥ = ⟨ x, x ⟩$ . These spaces are the natural arena for solving linear equations involving functions rather than finite tuples of numbers, and completeness is precisely the property that lets limiting/approximation arguments succeed. The subject matters because differential and integral operators, Fourier analysis, and quantum observables all live as linear maps between such spaces. A surprising fact: in infinite dimensions the closed unit ball is never compact (Riesz's lemma), which forces an entirely new toolkit — weak topologies, spectral theory, and compact operators — to replace the finite-dimensional intuition that "bounded + closed = compact."

The Core Insight

The single organizing principle is completeness married to linearity gives geometry and analysis simultaneously. In a Hilbert space the inner product encodes an exact notion of angle and orthogonal projection, making the space its own dual (Riesz Representation Theorem) and giving the best-approximation/projection theorem that drives least squares, Fourier series, and quantum measurement. In a general Banach space the inner product is gone, so duality becomes a genuinely separate object and the three pillars — Hahn–Banach, Open Mapping, Uniform Boundedness — replace orthogonality as the engines that make the theory run.

Formal Statement

Normed and Banach Spaces

A norm on a vector space $X$ over $K \in {R, C}$ is a map $∥ \cdot ∥ : X \to [0, \infty)$ with:

∥ x ∥ = 0 ⟺ x = 0, ∥ λ x ∥ = ∣ λ ∣ ∥ x ∥, ∥ x + y ∥ \leq ∥ x ∥ + ∥ y ∥

⟳These are positivity/definiteness, absolute homogeneity, and the triangle inequality; they make

d (x, y) = ∥ x - y ∥

a translation-invariant metric.

The pair $(X, ∥ \cdot ∥)$ is a Banach space if it is complete in the metric $d (x, y) = ∥ x - y ∥$ : every Cauchy sequence converges in $X$ . Here $X$ is the space, $λ \in K$ a scalar, and $x, y \in X$ vectors.

Inner Product and Hilbert Spaces

An inner product on $X$ over $C$ is a map $⟨ \cdot, \cdot ⟩ : X \times X \to C$ that is linear in the first argument, conjugate-symmetric, and positive-definite:

⟨ x, x ⟩ \geq 0, ⟨ x, x ⟩ = 0 ⟺ x = 0, ⟨ y, x ⟩ = \overline{⟨ x, y ⟩}

⟳Conjugate symmetry forces

⟨ x, x ⟩ \in R

; combined with sesquilinearity it determines the full form via the polarization identity.

The induced norm is $∥ x ∥ = ⟨ x, x ⟩^{1/2}$ . If $X$ is complete in this norm it is a Hilbert space $H$ . Two vectors are orthogonal, $x ⊥ y$ , when $⟨ x, y ⟩ = 0$ .

Cauchy–Schwarz and the Parallelogram Law

For all $x, y$ in an inner product space:

∣ ⟨ x, y ⟩ ∣ \leq ∥ x ∥ ∥ y ∥

⟳Expand

0 \leq ⟨ x - λ y, x - λ y ⟩

and minimize over

λ = ⟨ x, y ⟩ /∥ y ∥^{2}

∥ x + y ∥^{2} + ∥ x - y ∥^{2} = 2∥ x ∥^{2} + 2∥ y ∥^{2}

⟳Expand both squared norms via the inner product; cross terms cancel.

Equation [4] characterizes Hilbert spaces: a Banach norm comes from an inner product iff it satisfies the parallelogram law (Jordan–von Neumann), with the inner product recovered by polarization

⟨ x, y ⟩ = \frac{1}{4} k = 0 \sum 3 i^{k} ∥ x + i^{k} y ∥^{2}

⟳Substitute the four values

y \mapsto i^{k} y

into

∥ x + y ∥^{2} = ∥ x ∥^{2} + ∥ y ∥^{2} + 2 Re ⟨ x, y ⟩

and combine.

Hilbert Space Projection Theorem

Let $K \subseteq H$ be a nonempty closed convex set and $x \in H$ . Then there exists a unique $p \in K$ minimizing distance:

∥ x - p ∥ = k \in K min ∥ x - k ∥

⟳Take a minimizing sequence

k_{n}

; the parallelogram law shows

{k_{n}}

is Cauchy, completeness gives the limit, convexity keeps it in

K

, and strict convexity gives uniqueness.

When $K = M$ is a closed subspace, $p = P_{M} x$ is the orthogonal projection, characterized by $x - P_{M} x ⊥ M$ , and $H = M \oplus M^{⊥}$ .

Riesz Representation Theorem

Every continuous linear functional $φ \in H^{*}$ is represented by a unique $y_{φ} \in H$ :

φ (x) = ⟨ x, y_{φ} ⟩ for all x \in H, ∥ φ ∥_{H^{*}} = ∥ y_{φ} ∥_{H}

⟳If

φ = 0

take

y = 0

; otherwise

ker φ

is closed with

1

-dimensional orthogonal complement spanned by some

z

, and

y_{φ} = \overline{φ (z)} z /∥ z ∥^{2}

Thus a Hilbert space is isometrically anti-isomorphic to its own dual, $H ≅ H^{*}$ .

Why It Holds / Derivation

Projection Theorem via the Parallelogram Law

Let $K$ be closed and convex, $d = in f_{k \in K} ∥ x - k ∥$ , and pick $k_{n} \in K$ with $∥ x - k_{n} ∥ \to d$ .

Apply [4] to $u = x - k_{n}$ , $v = x - k_{m}$ :

∥ k_{n} - k_{m} ∥^{2} = 2∥ x - k_{n} ∥^{2} + 2∥ x - k_{m} ∥^{2} - 4 x - \frac{k _{n} + k _{m}}{2}^{2}

⟳Set

u - v = k_{m} - k_{n}

and

u + v = 2 x - (k_{n} + k_{m})

in the parallelogram identity.

By convexity $\frac{k _{n} + k _{m}}{2} \in K$ , so $x - \frac{k _{n} + k _{m}}{2} \geq d$ . Hence

∥ k_{n} - k_{m} ∥^{2} \leq 2∥ x - k_{n} ∥^{2} + 2∥ x - k_{m} ∥^{2} - 4 d^{2} n, m \to \infty 0

⟳Both

∥ x - k_{n} ∥^{2}, ∥ x - k_{m} ∥^{2} \to d^{2}

, so the right side

\to 4 d^{2} - 4 d^{2} = 0

So ${k_{n}}$ is Cauchy; by completeness $k_{n} \to p \in H$ , and closedness gives $p \in K$ with $∥ x - p ∥ = d$ .

Uniqueness: if $p, p^{'}$ both attain $d$ , step 1 with $k_{n} = p, k_{m} = p^{'}$ gives $∥ p - p^{'} ∥^{2} \leq 4 d^{2} - 4 d^{2} = 0$ .

Orthogonality (subspace case): for $m \in M$ and $t \in R$ , $∥ x - p - t m ∥^{2} \geq ∥ x - p ∥^{2}$ expands to $- 2 t Re ⟨ x - p, m ⟩ + t^{2} ∥ m ∥^{2} \geq 0$ for all $t$ , forcing $Re ⟨ x - p, m ⟩ = 0$ ; replacing $m$ by $im$ kills the imaginary part. $■$

The Three Pillars of Banach Space Theory

These hold in every Banach space and need no inner product.

Hahn–Banach: a bounded functional on a subspace $Y \subseteq X$ extends to all of $X$ with the same norm:

\exists Φ \in X^{*} : Φ ∣_{Y} = φ, ∥Φ ∥_{X^{*}} = ∥ φ ∥_{Y^{*}}

⟳Extend one dimension at a time controlling by a sublinear functional, then invoke Zorn's lemma for a maximal extension.

Uniform Boundedness (Banach–Steinhaus): a pointwise-bounded family ${T_{α}}$ of bounded operators on a Banach space is uniformly bounded:

α sup ∥ T_{α} x ∥ < \infty \forall x ⟹ α sup ∥ T_{α} ∥ < \infty

⟳Baire category — one of the closed sets

{x : sup_{α} ∥ T_{α} x ∥ \leq n}

has nonempty interior.

Open Mapping / Closed Graph: a bounded surjection between Banach spaces is open; equivalently a closed-graph linear map is bounded.

T \in B (X, Y) surjective ⟹ T open; Γ (T) closed ⟹ T bounded

⟳Baire category gives

T (B_{X})

dense and "fat"; a completeness/iteration argument upgrades density to an open neighborhood of

0

■

Stefan Banach | 1892–1945

Banach, largely self-taught and working in Lwów, axiomatized complete normed spaces in his 1922 thesis and his 1932 monograph Théorie des opérations linéaires, which gave the field its name and its central theorems (open mapping, closed graph, Hahn–Banach in its analytic form). He led the legendary Scottish Café seminar, where problems were recorded in the Scottish Book. His insistence on abstract axioms over concrete function spaces — earlier developed by Fréchet, F. Riesz, and Hilbert — turned a collection of techniques into a coherent geometric theory.

Building Intuition

Think of a Hilbert space as $R^{n}$ with $n = \infty$ but the geometry intact: you can still drop a perpendicular, take an orthonormal basis, expand any vector in coordinates (Fourier coefficients), and Pythagoras holds. The closed-subspace projection is literally "the foot of the perpendicular," which is why least-squares regression, conditional expectation in probability, and quantum measurement are all the same theorem.

A general Banach space is what you get when you keep distances but throw away angles. You can still measure lengths and converge, but there is no canonical perpendicular, so "the closest point in a subspace" may not be unique and the space need not equal $M \oplus M^{⊥}$ . This is why $ℓ^{p}$ and $L^{p}$ for $p \neq = 2$ feel "bent": their unit balls have corners or flat faces rather than being round, and that lost roundness is exactly the failed parallelogram law [4].

Unit balls: roundness of $\ell^2$ vs. corners/flatness of $\ell^1$ and $\ell^\infty$, illustrating why only $p=2$ is a Hilbert space

Worked Examples

Best Approximation by Fourier Coefficients in $L^2$

Given:

H = L^{2} ([- π, π])

with

⟨ f, g ⟩ = \int_{- π}^{π} f \overset{g}{ˉ} d x

, orthonormal system

e_{n} (x) = \frac{1}{2 π} e^{in x}

, and target

f (x) = x

. Approximate

f

in the subspace

M = span {e_{- 1}, e_{0}, e_{1}}

. Steps:

The projection is $P_{M} f = \sum_{n = - 1}^{1} ⟨ f, e_{n} ⟩ e_{n}$ (orthonormal coordinates).
Compute coefficients. $⟨ x, e_{0} ⟩ = \frac{1}{2 π} \int_{- π}^{π} x d x = 0$ .
For $n = \pm 1$ : $⟨ x, e_{n} ⟩ = \frac{1}{2 π} \int_{- π}^{π} x e^{- in x} d x$ . Integrating by parts, $\int_{- π}^{π} x e^{- in x} d x = \frac{2 π}{n} (- 1)^{n + 1} i$ ... giving $⟨ x, e_{1} ⟩ = i 2 π \cdot (- 1)^{2} /1 = i 2 π$ up to sign, and $⟨ x, e_{- 1} ⟩ = - i 2 π$ .
Recombining the $\pm 1$ terms yields the real trigonometric form $P_{M} f (x) = 2 sin x$ (the first Fourier sine term of $x$ ).
The residual $f - P_{M} f$ is orthogonal to $M$ by [6].

Answer:

P_{M} f (x) = 2 sin x

— the unique closest function in

M

f (x) = x

L^{2}

-norm.

Operator Norm and Spectrum of a Multiplication Operator

Given: On

H = L^{2} ([0, 1])

define

(T f) (x) = x f (x)

. Steps:

Boundedness: $∥ T f ∥^{2} = \int_{0}^{1} x^{2} ∣ f ∣^{2} d x \leq \int_{0}^{1} ∣ f ∣^{2} d x = ∥ f ∥^{2}$ , so $∥ T ∥ \leq 1$ .
Sharpness: take $f_{n}$ concentrated near $x = 1$ ; then $∥ T f_{n} ∥/∥ f_{n} ∥ \to 1$ , so $∥ T ∥ = 1$ .
Self-adjointness: $⟨ T f, g ⟩ = \int_{0}^{1} x f \overset{g}{ˉ} = ⟨ f, T g ⟩$ , hence $T = T^{*}$ .
Eigenvalues: $T f = λ f$ means $(x - λ) f = 0$ a.e., forcing $f = 0$ — no eigenvalues.
Spectrum: $T - λ I$ is invertible iff $\frac{1}{x - λ}$ is bounded on $[0, 1]$ , i.e. iff $λ \in / [0, 1]$ . So $σ (T) = [0, 1]$ .

Answer:

∥ T ∥ = 1, σ (T) = [0, 1], point spectrum = \emptyset

— a self-adjoint operator with purely continuous spectrum, impossible in finite dimensions.

Common Misconceptions

"Bounded and closed implies compact." False in infinite dimensions. By Riesz's lemma, the closed unit ball of a normed space is compact iff the space is finite-dimensional. This is why we need weak/weak-* topologies (Banach–Alaoglu) to recover any compactness, and why "compact operator" is a special, restrictive class rather than the norm. "Every closed subspace has an orthogonal complement / every functional is an inner product." True in Hilbert spaces, false in general Banach spaces. The Riesz representation [7] and the decomposition

H = M \oplus M^{⊥}

are Hilbert-only luxuries. In

ℓ^{1}

C [0, 1]

, not every closed subspace is complemented, and the dual is a genuinely different space (

(ℓ^{1})^{*} = ℓ^{\infty}

(c_{0})^{*} = ℓ^{1}

A Discontinuous Linear Functional and Non-Convergent Fourier Series

On the incomplete space of finitely-supported sequences with $ℓ^{2}$ -norm, the functional $φ (x) = \sum_{n} n x_{n}$ is linear but unbounded ( $φ (e_{n}) = n$ while $∥ e_{n} ∥ = 1$ ). Such pathologies require the axiom of choice to exhibit on complete spaces — consistent with the existence of a Hamel basis.

Separately, Du Bois-Reymond's example (1873) gives a continuous function whose Fourier series diverges at a point: pointwise convergence fails even for continuous data. This is invisible in $L^{2}$ , where Fourier series always converge in norm by Orthonormal Bases & Bessel's Inequality — a vivid demonstration that the choice of norm changes which theorems are true.

Connections

Functional analysis sits at the crossroads of several theories. The spectral theorem for self-adjoint and compact operators (Spectral Theory of Operators) generalizes the diagonalization of symmetric matrices and underlies Quantum Mechanics Mathematical Foundations, where observables are self-adjoint operators and states are unit vectors. The duality theory connects to Locally Convex & Topological Vector Spaces (where weak topologies and Hahn–Banach separation live) and to Distributions & Sobolev Spaces, the completed function spaces in which modern PDE theory operates. The projection theorem [6] is the abstract heart of Conditional Expectation & Martingales in probability and of Least Squares & Variational Methods. Finally, Fourier Analysis & Harmonic Analysis is essentially the spectral theory of translation-invariant operators on $L^{2}$ .

Real-World Applications

Quantum mechanics. States are unit vectors in a complex Hilbert space; observables are self-adjoint operators; the spectral theorem gives measurement outcomes and the Born rule probabilities $∣ ⟨ ψ, e_{n} ⟩ ∣^{2}$ .
Signal processing & JPEG/MP3 compression. Orthonormal-basis expansions (Fourier, wavelets) and best $k$ -term approximation are projection-theorem applications; truncating small coefficients is $L^{2}$ -optimal.
Numerical PDE — finite elements. The Lax–Milgram theorem (a coercive bilinear form on a Hilbert space) and Galerkin projection turn weak formulations of PDEs into solvable finite-dimensional systems.
Machine learning — kernel methods/SVM. Reproducing kernel Hilbert spaces (RKHS) make nonlinear classification a linear projection in a feature space; the representer theorem is a Riesz-type result.
Statistics — least squares & regression. The normal equations are exactly orthogonal projection onto the column space of the design matrix.
Control theory. $H^{\infty}$ and $H^{2}$ optimization formulate robust controller design as norm minimization in Hardy (Banach/Hilbert) spaces of analytic functions.
Tomography / inverse problems. Compact operators and their singular value decompositions (a Hilbert-space SVD) quantify ill-posedness and drive Tikhonov regularization.

Historical Development

1900–1906: Hilbert and Schmidt study integral equations and infinite quadratic forms, introducing $ℓ^{2}$ "Hilbert space" implicitly via spectral ideas.
1906: Fréchet's thesis introduces abstract metric spaces, the framework for completeness.
1907: F. Riesz and Fischer prove the Riesz–Fischer theorem ( $L^{2}$ completeness and Fourier-coefficient correspondence), and Riesz proves the representation theorem for $L^{2}$ .
1910–1918: Riesz develops $L^{p}$ and $ℓ^{p}$ spaces and bounded operators; F. Riesz proves the lemma on non-compactness of the unit ball.
1922: Banach's doctoral thesis axiomatizes complete normed spaces ("spaces of type (B)"); Wiener independently formulates similar ideas.
1927: Hahn, and independently Banach (1929), prove the analytic Hahn–Banach extension theorem.
1929–1932: von Neumann gives the axiomatic definition of (abstract, separable) Hilbert space and the spectral theorem for unbounded self-adjoint operators, motivated by quantum mechanics.
1932: Banach's Théorie des opérations linéaires unifies the field; Banach–Steinhaus and the open mapping theorem appear.
1934–1940s: Alaoglu (weak-* compactness, 1940), Gelfand (Banach algebras, late 1930s), and Eidelheit/Mazur (separation theorems) round out the structural theory.

Subtleties & Limits

Several deep structural questions remain. The invariant subspace problem — does every bounded operator on a separable infinite-dimensional Hilbert space have a nontrivial closed invariant subspace? — is still open for Hilbert spaces, though Enflo (1987) and Read constructed Banach-space counterexamples. Enflo (1973) also resolved the approximation problem negatively (a Banach space without the approximation property), refuting a long-standing conjecture. The complemented subspace problem (only Hilbert spaces have all closed subspaces complemented) was settled by Lindenstrauss–Tzafriri (1971). Many isometric/isomorphic classification questions for $L^{p}$ and $C (K)$ spaces remain delicate, and the geometry of Banach spaces (type, cotype, finite representability) is an active research area linking probability, combinatorics, and operator theory.