Linear algebra & the spectral theorem

One-line statement: Every self-adjoint (Hermitian) operator on a finite-dimensional inner product space has an orthonormal basis of eigenvectors, and is therefore diagonalizable by a unitary change of coordinates with real eigenvalues. Key result in words: Symmetric/Hermitian and, more generally, normal matrices are exactly those that can be diagonalized in an orthonormal basis; the geometry of the operator decomposes into independent stretching along mutually perpendicular axes. Where it applies: Quantum mechanics (observables), principal component analysis, vibration analysis, quadratic forms, Markov chains, differential equations, and any setting reducible to symmetric/Hermitian operators. Who/when: Roots in Cauchy (1829), Hermite, Jacobi; rigorous infinite-dimensional version by Hilbert, von Neumann (1929–1932).

The spectral theorem is the crowning structural result of finite-dimensional linear algebra: it states precisely which linear operators can be diagonalized using an orthonormal basis, and identifies that class as the normal operators (those commuting with their adjoint), with the self-adjoint operators forming the important real-eigenvalue subclass. Concretely, for a real symmetric matrix $A = A^{T}$ there exists an orthogonal matrix $Q$ such that $Q^{T} A Q = Λ$ is diagonal, and for a complex Hermitian matrix $A = A^{*}$ there exists a unitary $U$ with $U^{*} A U = Λ$ with real entries. This matters because it converts a coupled, geometric, or physical problem into $n$ decoupled one-dimensional problems along perpendicular axes — the eigenvectors. A surprising fact: the same algebraic condition $A^{*} A = A A^{*}$ that looks merely technical is exactly the necessary-and-sufficient condition for the existence of a clean orthonormal eigenbasis, so unitary, Hermitian, and skew-Hermitian matrices are all diagonalizable for one unified reason.

The Core Insight

A general linear map shears, rotates, and stretches space in entangled ways. But if the operator respects the inner product in the right way — if it is normal — then space splits into mutually orthogonal one-dimensional (or invariant) pieces on each of which the operator acts as pure scalar multiplication. The whole operator is then nothing but a weighted sum of orthogonal projections. Diagonalization with an orthonormal basis is geometry, not just bookkeeping: the eigenvectors are genuinely perpendicular directions of pure scaling.

Formal Statement

Inner product space, adjoint, normal, self-adjoint

Let $V$ be a finite-dimensional vector space over $F \in {R, C}$ with inner product $⟨ \cdot, \cdot ⟩$ (conjugate-linear in the first or second slot per convention; here linear in the second, conjugate-linear in the first).

The adjoint $T^{*}$ of $T : V \to V$ is the unique operator satisfying $⟨ T x, y ⟩ = ⟨ x, T^{*} y ⟩$ for all $x, y$ .
$T$ is self-adjoint (Hermitian) if $T = T^{*}$ ; in matrix terms $A = A^{*}$ (conjugate transpose), and over $R$ , $A = A^{T}$ (symmetric).
$T$ is unitary if $T^{*} T = T T^{*} = I$ (orthogonal over $R$ ).
$T$ is normal if $T^{*} T = T T^{*}$ .
$T$ is skew-adjoint if $T^{*} = - T$ .

Self-adjoint, unitary, and skew-adjoint operators are all normal.

⟨ T x, y ⟩ = ⟨ x, T^{*} y ⟩ \forall x, y \in V .

⟳This is the defining property of the adjoint; existence/uniqueness follow from the Riesz representation theorem applied to the linear functional

x \mapsto ⟨ T x, y ⟩

A = A^{*} ⟺ a_{ij} = \overline{a_{j i}} (Hermitian); A = A^{T} ⟺ a_{ij} = a_{j i} (real symmetric) .

⟳Express

⟨ T x, y ⟩ = ⟨ x, T y ⟩

in coordinates with

⟨ u, v ⟩ = u^{*} v

; equating entries gives the matrix condition.

Spectral Theorem — Complex (Normal) Version

Let $V$ be a finite-dimensional complex inner product space and $T : V \to V$ linear. Then $T$ is normal ( $T T^{*} = T^{*} T$ ) if and only if there is an orthonormal basis ${u_{1}, \dots, u_{n}}$ of $V$ consisting of eigenvectors of $T$ . Equivalently, there exists a unitary $U$ and diagonal $Λ$ with

A = U Λ U^{*} = k = 1 \sum n λ_{k} u_{k} u_{k}^{*} .

⟳

U = [u_{1} \dots u_{n}]

has orthonormal columns so

U^{*} U = I

;

A U = U Λ

encodes

A u_{k} = λ_{k} u_{k}

; right-multiply by

U^{*}

Spectral Theorem — Real Symmetric Version

Let $A$ be a real $n \times n$ symmetric matrix ( $A = A^{T}$ ). Then all eigenvalues of $A$ are real, and there exists an orthogonal matrix $Q$ ( $Q^{T} Q = I$ ) such that

Q^{T} A Q = Λ = diag (λ_{1}, \dots, λ_{n}), λ_{k} \in R .

⟳Symmetric real matrices are Hermitian when viewed over

C

; reality of eigenvalues plus the ability to choose real eigenvectors forces a real orthogonal diagonalizer.

Spectral Decomposition / Resolution of Identity

If $T$ is normal with distinct eigenvalues $μ_{1}, \dots, μ_{m}$ and $P_{j}$ is the orthogonal projection onto the eigenspace $E_{μ_{j}} = ker (T - μ_{j} I)$ , then

T = j = 1 \sum m μ_{j} P_{j}, j = 1 \sum m P_{j} = I, P_{j} P_{k} = δ_{j k} P_{j}, P_{j}^{*} = P_{j} .

⟳Group the rank-one terms

λ_{k} u_{k} u_{k}^{*}

in [3] by equal eigenvalue; each group sums to an orthogonal projection onto an eigenspace.

Why It Holds / Derivation

Spectral Theorem via Schur + Normality

We prove the complex normal version in numbered steps.

Step 1 (Schur triangularization). Over

C

every operator has at least one eigenvalue (fundamental theorem of algebra applied to

det (A - λ I) = 0

). By induction on

dim V

, choosing a unit eigenvector and restricting

T

to the orthogonal complement of its span (after orthogonal projection), one builds an orthonormal basis in which

T

is upper triangular:

U^{*} A U = R, R upper triangular, U unitary .

⟳Extend a unit eigenvector to an orthonormal basis; the first column of

U^{*} A U

λ e_{1}

; recurse on the trailing

(n - 1) \times (n - 1)

block.

Step 2 (Normality is preserved). Since

U

is unitary,

R = U^{*} A U

satisfies

R^{*} R = U^{*} A^{*} A U

and

R R^{*} = U^{*} A A^{*} U

. Because

A

is normal,

A^{*} A = A A^{*}

, hence

R^{*} R = R R^{*}

R

is normal. Step 3 (Normal + triangular ⇒ diagonal). Compare diagonal entries of

R^{*} R

and

R R^{*}

. For the

(1, 1)

entry,

(R R^{*})_{11} = k \sum ∣ r_{1 k} ∣^{2}, (R^{*} R)_{11} = ∣ r_{11} ∣^{2} .

⟳

R

upper triangular means row 1 may have many entries but column 1 has only

r_{11}

; write out the matrix products entrywise.

Equality forces $r_{1 k} = 0$ for $k > 1$ . Proceeding down the diagonal inductively kills every off-diagonal entry, so $R$ is diagonal. Thus $A = U Λ U^{*}$ , giving an orthonormal eigenbasis — the columns of $U$ .

Step 4 (Converse). If

A = U Λ U^{*}

, then

A^{*} = U Λ^{*} U^{*}

and since diagonal matrices commute,

A^{*} A = U Λ^{*} Λ U^{*} = U Λ Λ^{*} U^{*} = A A^{*}

, so

A

is normal.

■

Eigenvalues of Self-Adjoint Operators are Real; Eigenvectors Orthogonal

Let $T = T^{*}$ , $T x = λ x$ with $x \neq = 0$ .

λ ⟨ x, x ⟩ = ⟨ T x, x ⟩ = ⟨ x, T^{*} x ⟩ = ⟨ x, T x ⟩ = \overline{λ} ⟨ x, x ⟩ .

⟳Use [1] with

y = x

and self-adjointness; since

⟨ x, x ⟩ > 0

, conclude

λ = \overline{λ}

Hence $λ = \overline{λ} \in R$ . For distinct eigenvalues $λ \neq = μ$ with $T x = λ x$ , $T y = μ y$ :

λ ⟨ y, x ⟩ = ⟨ y, T x ⟩ = ⟨ T y, x ⟩ = μ ⟨ y, x ⟩ \Rightarrow (λ - μ) ⟨ y, x ⟩ = 0 \Rightarrow ⟨ y, x ⟩ = 0.

⟳Move

T

across the inner product using self-adjointness and reality of

μ

; nonzero

λ - μ

forces orthogonality.

■

John von Neumann | 1903–1957

Von Neumann gave quantum mechanics its rigorous mathematical home in Mathematische Grundlagen der Quantenmechanik (1932), generalizing the finite-dimensional spectral theorem to unbounded self-adjoint operators on infinite-dimensional Hilbert space via the spectral measure $E (\cdot)$ , writing $T = \int_{R} λ d E (λ)$ . In his framework physical observables are self-adjoint operators, their possible measured values are the spectrum, and the projection-valued measure encodes the probabilities of outcomes — the infinite-dimensional shadow of equation [5]. He also unified the Heisenberg matrix mechanics and Schrödinger wave mechanics by showing both act on the same Hilbert space.

Building Intuition

Think of a symmetric matrix as the quadratic form $q (x) = x^{T} A x$ , e.g. the equation of an ellipse or ellipsoid. The level set $q (x) = 1$ is a tilted ellipse; the spectral theorem says you can rotate your head (an orthogonal change of coordinates) until the ellipse's axes line up with the coordinate axes. In the new coordinates $q = \sum_{k} λ_{k} y_{k}^{2}$ — pure squares, no cross terms. The eigenvectors are the principal axes and the eigenvalues are $1/ (semi-axis)^{2}$ . Positive eigenvalues ⇒ ellipsoid, mixed signs ⇒ saddle (hyperboloid), a zero eigenvalue ⇒ a flat (degenerate) direction. Because $A$ is symmetric the axes are guaranteed perpendicular; a non-symmetric matrix can have eigenvectors at oblique angles, and you'd need a shear, not a rotation, to diagonalize it.

A real symmetric operator rotates to principal axes; the ellipse $x^TAx=1$ has perpendicular eigen-axes scaled by $1/\sqrt{\lambda_i}$.

<svg viewBox="0 0 460 320" xmlns="http://www.w3.org/2000/svg">
<rect width="460" height="320" fill="#0a0e17"/>
<line x1="40" y1="160" x2="420" y2="160" stroke="#94a3b8" stroke-width="1"/>
<line x1="230" y1="20" x2="230" y2="300" stroke="#94a3b8" stroke-width="1"/>
<text x="424" y="164" fill="#e2e8f0" font-size="12">x</text>
<text x="234" y="20" fill="#e2e8f0" font-size="12">y</text>
<ellipse cx="230" cy="160" rx="130" ry="60" transform="rotate(-30 230 160)" fill="none" stroke="#d4a853" stroke-width="2.5"/>
<text x="300" y="80" fill="#d4a853" font-size="12">xᵀAx = 1</text>
<line x1="230" y1="160" x2="343" y2="95" stroke="#e2e8f0" stroke-width="2"/>
<line x1="230" y1="160" x2="200" y2="108" stroke="#e2e8f0" stroke-width="2"/>
<text x="348" y="92" fill="#e2e8f0" font-size="12">u₁ (λ₁)</text>
<text x="150" y="104" fill="#e2e8f0" font-size="12">u₂ (λ₂)</text>
<text x="60" y="40" fill="#e2e8f0" font-size="11">semi-axes = 1/√λᵢ</text>
<text x="60" y="296" fill="#94a3b8" font-size="11">u₁ ⊥ u₂  (orthonormal eigenbasis)</text>
</svg>

Worked Examples

Diagonalizing a 2×2 real symmetric matrix

Given:

A = (2112) = A^{T}

. Step 1 — Characteristic polynomial.

det (A - λ I) = (2 - λ)^{2} - 1 = λ^{2} - 4 λ + 3 = (λ - 1) (λ - 3) .

⟳Expand the determinant of

(2 - λ 1 1 2 - λ)

So $λ_{1} = 3, λ_{2} = 1$ (both real, as guaranteed).

Step 2 — Eigenvectors. For

λ_{1} = 3

(A - 3 I) v = 0 \Rightarrow (- 1 1 1 - 1) v = 0 \Rightarrow v_{1} = (1, 1)

. For

λ_{2} = 1

v_{2} = (1, - 1)

. These are orthogonal:

v_{1} \cdot v_{2} = 0

. ✓ Step 3 — Normalize and form $Q$ .

Q = \frac{1}{2} (11 1 - 1), Q^{T} A Q = (3001) .

⟳Divide each eigenvector by its norm

2

; verify

Q^{T} A Q

by direct multiplication.

Boxed answer:

A = Q Λ Q^{T}, Λ = diag (3, 1), Q = \frac{1}{2} (11 1 - 1) .

The quadratic form becomes

3 y_{1}^{2} + y_{2}^{2}

Spectral projections and a function of a matrix

Given: Same

A

as above. Compute

A

(positive square root) and verify the projection decomposition. Step 1 — Projections. With

u_{1} = \frac{1}{2} (1, 1)^{T}

u_{2} = \frac{1}{2} (1, - 1)^{T}

P_{1} = u_{1} u_{1}^{T} = \frac{1}{2} (1111), P_{2} = u_{2} u_{2}^{T} = \frac{1}{2} (1 - 1 - 1 1) .

⟳Outer product of each unit eigenvector with itself; check

P_{1} + P_{2} = I

and

P_{1} P_{2} = 0

Step 2 — Spectral form.

A = 3 P_{1} + 1 \cdot P_{2}

. Any function applies to eigenvalues:

f (A) = f (3) P_{1} + f (1) P_{2} .

⟳

f (A) = Q f (Λ) Q^{T}

; grouping by eigenvalue reproduces the projection sum [5].

Step 3 — Square root.

f (λ) = λ

A = 3 P_{1} + 1 \cdot P_{2} = \frac{1}{2} (3 + 1 3 - 1 3 - 1 3 + 1) .

⟳Plug

f (3) = 3, f (1) = 1

into [13] using the matrices [12].

Boxed answer:

A \approx (1.366 0.366 0.366 1.366),

and one checks

(A)^{2} = A

exactly. (Units: dimensionless, as a pure matrix.)

Common Misconceptions

"Every diagonalizable matrix has orthogonal eigenvectors." False.

A = (1012)

is diagonalizable (distinct eigenvalues

1, 2

) but its eigenvectors

(1, 0)

and

(1, 1)

are not orthogonal. Orthonormal diagonalizability is the special privilege of normal matrices. Also beware: "Hermitian ⇒ positive eigenvalues" is wrong — Hermitian only guarantees real eigenvalues; positivity requires the stronger condition

x^{*} A x > 0

(positive definite).

Defective non-normal matrix

The matrix $N = (0010)$ satisfies $N^{2} = 0$ but $N \neq = 0$ . Its only eigenvalue is $0$ with a single one-dimensional eigenspace — there is no basis of eigenvectors at all, let alone an orthonormal one. Check normality:

N^{*} N = (0001) \neq = (1000) = N N^{*} .

⟳Compute both products directly; their inequality confirms

N

is not normal, consistent with its non-diagonalizability.

This shows the spectral theorem genuinely fails outside the normal class: such defective matrices need the Jordan Normal Form instead.

Connections

The spectral theorem is the symmetric heart of several broader results. The Singular Value Decomposition generalizes it to arbitrary (even rectangular) matrices via $A = U Σ V^{*}$ , obtained by spectrally decomposing the symmetric $A^{*} A$ and $A A^{*}$ . The Principal Axis Theorem for quadratic forms is exactly the real symmetric case. The Cayley–Hamilton Theorem and minimal polynomials explain when diagonalization (with any basis) is possible, while the Jordan Normal Form handles the defective remainder. Over function spaces it extends to compact self-adjoint operators (Hilbert Space theory) giving eigenfunction expansions, and to unbounded operators underpinning Quantum Mechanics. The notion of Positive Definite Matrices is exactly "self-adjoint with positive spectrum."

Real-World Applications

Principal Component Analysis (PCA): Diagonalize the symmetric covariance matrix $Σ = \frac{1}{n} X^{T} X$ ; eigenvectors are principal directions, eigenvalues are variances — the backbone of dimensionality reduction in statistics and machine learning.
Quantum mechanics: Observables (energy, momentum, spin) are self-adjoint operators; their real spectra are the measurable values, and orthonormal eigenstates give measurement probabilities via $∣ ⟨ u_{k}, ψ ⟩ ∣^{2}$ .
Structural & mechanical vibration: The stiffness/mass system $M \overset{x}{¨} = - K x$ with symmetric $K, M$ decouples into normal modes — the generalized eigenvectors — predicting resonance frequencies of bridges and buildings.
Google PageRank & Markov chains: Stationary distributions are eigenvectors; for reversible chains a symmetrization makes the spectral theorem give real, ordered eigenvalues controlling mixing time.
Image compression (low-rank approximation): Truncating the spectral/SVD expansion keeps the dominant eigenmodes, compressing data with minimal error (Eckart–Young).
Stability of equilibria: The Hessian of a potential energy is symmetric; its eigenvalues' signs classify minima, maxima, and saddle points.
Quantum chemistry / molecular orbitals: Diagonalizing the Hermitian Fock/Hamiltonian matrix yields orbital energies and orthonormal molecular orbitals.

Historical Development

1829 — Cauchy proves that a real symmetric (quadratic-form) matrix has real eigenvalues and orthogonal principal axes, in the context of the rotation of rigid bodies and conic sections.
1846 — Jacobi develops iterative rotation methods to diagonalize symmetric matrices (the Jacobi eigenvalue algorithm), giving constructive force to the theorem.
1855 — Hermite introduces what are now called Hermitian forms and proves reality of their eigenvalues, extending Cauchy's result to complex coefficients.
1858 — Cayley lays out matrix algebra; the characteristic equation and Cayley–Hamilton theorem clarify eigenvalue structure.
1870s–1880s — Weierstrass & Frobenius systematize elementary divisors and canonical forms, sharply distinguishing diagonalizable from defective matrices.
1904–1910 — Hilbert develops the spectral theory of infinite systems and integral equations, coining "spectrum" and treating quadratic forms in infinitely many variables.
1929 — von Neumann proves the spectral theorem for unbounded self-adjoint operators via projection-valued measures.
1932 — von Neumann's Mathematical Foundations of Quantum Mechanics canonizes the operator/observable correspondence.
1965 — Golub & Kahan / Francis (QR algorithm) make numerical spectral decomposition fast and stable, the standard in scientific computing today.

Subtleties & Limits

The clean finite-dimensional statement frays at the edges. (1) Infinite dimensions: a self-adjoint operator may have continuous spectrum with no eigenvectors at all (e.g. the position operator $x^$ on $L^{2} (R)$ ), so the sum [5] must become an integral against a projection-valued measure, and unbounded operators force delicate domain considerations. (2) Numerical sensitivity: eigenvectors of matrices with near-degenerate eigenvalues are ill-conditioned, and clustered spectra are computationally hard to separate; quantifying optimal algorithms for the symmetric eigenvalue problem under finite precision remains an active area. (3) Joint diagonalization: several non-commuting normal matrices generally have no common eigenbasis (the algebraic root of the Heisenberg uncertainty principle), and finding approximate joint diagonalizers (used in blind source separation) is an open optimization frontier. (4) Non-normal operators governing transient growth in fluid dynamics are not captured by eigenvalues alone — pseudospectra are needed — showing the spectral theorem's reach ends precisely at non-normality.