Variational Principles for Eigenvalue Problems

Solving an eigenvalue problem the direct way — write down the differential operator, impose the boundary conditions, and hunt for the values of $\lambda$ that admit a non-trivial solution — is often hopeless. The variational principle offers a different route. It recasts $Lu=\lambda u$ as extremisation: the eigenvalues are the stationary values of a single scalar functional, the Rayleigh quotient $R[u]=\langle u,Lu\rangle/\langle u,u\rangle$ , and the eigenfunctions are the functions where it is stationary. This is the eigenvalue analogue of Hamilton's principle of stationary action, governed by the machinery of the first variation and stationarity developed earlier in this chapter. One structural fact does all the work. For a self-adjoint operator the gradient of the Rayleigh quotient vanishes precisely at the eigenfunctions, where the quotient equals the corresponding eigenvalue. Three consequences follow. First, the ground-state eigenvalue is the global minimum, so any normalised trial function delivers a rigorous upper bound — the workhorse of quantum chemistry and nuclear physics. Second, higher eigenvalues follow by minimising over subspaces orthogonal to the lower modes, or, free of that, through the Courant–Fischer minimax theorem. Third, restricting to a finite-dimensional span converts the continuous problem into a generalised matrix eigenproblem whose roots interlace and bound the true spectrum — the Rayleigh–Ritz method, and through it the finite element method and most of computational quantum mechanics, producing bounds that improve monotonically as the basis grows.

Key takeaways

The eigenvalues of a self-adjoint $L$ are the stationary values of the Rayleigh quotient $R[u]=\langle u,Lu\rangle/\langle u,u\rangle$ ; the eigenfunctions are where $R$ is stationary.
The lowest eigenvalue is a genuine minimum, $\lambda_1=\min_u R[u]$ : every admissible trial function is an upper bound, and the error in $\lambda$ is quadratic in the trial-function error.
Higher eigenvalues follow from orthogonality constraints, or free of them from the Courant–Fischer minimax principle $\lambda_n=\min_{\dim V=n}\max_{u\in V}R[u]$ .
For a Sturm–Liouville operator the quotient is $R[u]=\int(pu'^2+qu^2)\,\mathrm{d}x/\int wu^2\,\mathrm{d}x$ , with boundary terms treated according to the imposed self-adjoint boundary conditions; stationarity reproduces the eigenvalue equation, and where the endpoint values are free the variational principle also yields the corresponding natural boundary conditions.
A finite basis turns the problem into the generalised matrix eigenproblem $H\mathbf{c}=\lambda S\mathbf{c}$ — the Rayleigh–Ritz method behind Hartree–Fock and finite elements.
Ritz estimates decrease monotonically as the basis grows and interlace the true spectrum, so refinement only sharpens them.
Key idea: for a self-adjoint operator $L$ the eigenvalues are exactly the stationary values of the Rayleigh quotient $R[u]=\langle u,Lu\rangle/\langle u,u\rangle$ ; the lowest is a true minimum, so every trial function bounds the ground state from above.

The core insight

For a self-adjoint operator $L$ , the eigenvalues are exactly the stationary values of the Rayleigh quotient $R[u]=\langle u,Lu\rangle/\langle u,u\rangle$ ; the lowest is a genuine minimum, so every admissible trial function bounds the ground state from above, with an error only quadratic in the trial function's error.

Where this lives today

The variational principle is the live engine of computational physics and chemistry. Every mainstream electronic-structure method — Hartree–Fock, configuration interaction, Kohn–Sham density-functional theory — is a Rayleigh–Ritz calculation in disguise: molecular orbitals expanded in a finite atomic-orbital basis, the energy stationarity condition becoming the Roothaan generalised eigenproblem $F\mathbf c=\varepsilon S\mathbf c$ , with $S$ the overlap matrix precisely because atomic orbitals are non-orthogonal. Enlarging the basis lowers the energy monotonically — "basis-set convergence" is the monotone-convergence theorem measured in kilojoules per mole. In condensed matter and nuclear physics, variational Monte Carlo optimises Jastrow, BCS, and Gutzwiller trial states by direct minimisation, reaching ground states no exact method can. In engineering, the finite element method — Rayleigh–Ritz with piecewise-polynomial shape functions on a mesh — computes turbine-blade vibration modes and aircraft-frame buckling loads as bounds that tighten under mesh refinement. Even quantum computing leans on it: the variational quantum eigensolver prepares a trial state on a processor and minimises its measured Rayleigh quotient classically — the oldest idea in the subject on the newest hardware.

The physical picture

Read $R[u]$ as energy per unit norm: the expected value of $L$ in the state $u$ , divided by the state's size. Nature settles into the configuration that makes this ratio smallest at fixed normalisation, so the minimum of the quotient is the ground-state energy and the minimiser is the ground-state mode. Everything else — bounds, the matrix method, convergence — follows from that identification. The cleanest way to see the whole spectrum is to walk a straight line in function space. Mix two true eigenfunctions, $u=u_1\cos\theta+u_2\sin\theta$ ; being orthogonal eigenfunctions, the quotient collapses to $R=\lambda_1\cos^2\theta+\lambda_2\sin^2\theta=\lambda_1+(\lambda_2-\lambda_1)\sin^2\theta$ — a smooth curve pinned to $\lambda_1$ and $\lambda_2$ at its ends and flat at both. That flatness at the ground eigenfunction, at $\theta=0$ and at $\theta=\pi/2$ , is why variational estimates work: for a normalised trial function close to the ground state, the error in the variational energy is second order in the wavefunction error.

Figure

The Rayleigh quotient along a line in function space, u = u₁cosθ + u₂sinθ, gives R = λ₁ + (λ₂ − λ₁)sin²θ for the particle-in-a-box levels λ₁ = π² = 9.87 and λ₂ = 4π² = 39.48; R is stationary and minimal at the ground eigenfunction θ = 0, and a tilted trial at θ = 20° gives R = 13.3, safely above λ₁

Now let an arbitrary trial function loose. Expand it in the true eigenbasis, $u=\sum_n c_n u_n$ ; the quotient becomes $R[u]=\sum_n\lambda_n|c_n|^2\big/\sum_n|c_n|^2$ , a weighted average of the eigenvalues with non-negative weights. An average can never dip below its smallest member, so $R[u]\ge\lambda_1$ for every $u$ — the variational bound, with no calculus at all. The second figure shows the payoff: a one-parameter hydrogen trial whose energy $E(\alpha)$ we minimise to pin the ground state exactly.

Figure

The hydrogen variational energy E(α) = α²/2 − α for the trial ψ = exp(−αr) in Hartree atomic units, minimised at α = 1 where E = −0.5 Ha = −13.6 eV; the curve is flat at its minimum, so a 1% error in α shifts E by only about 0.01%

The core equations

The whole subject hangs on one object: the Rayleigh quotient assigns to every admissible function the expectation of $L$ in the state $u$ , normalised by the state's norm in the inner product that makes $L$ self-adjoint.

R[u]=\frac{\langle u,Lu\rangle}{\langle u,u\rangle},\qquad \langle f,g\rangle=\int_a^b \bar f\,g\,w\,\mathrm{d}x \qquad (1)

where: $L$ — self-adjoint linear operator; $u$ — trial function in the form domain of $L$ ; $w(x)>0$ — weight defining the inner product; $\langle\cdot,\cdot\rangle$ — weighted inner product on $[a,b]$ . Reading: $R[u]$ is scale-invariant — replacing $u$ by $cu$ leaves it unchanged — so normalisation is a convenience. The central theorem: the stationary points of this functional are exactly the eigenfunctions, the stationary values exactly the eigenvalues — vanishing first variation and the eigenequation are one statement.

\delta R[u]=0 \quad\Longleftrightarrow\quad Lu=\lambda u, \qquad \lambda=R[u] \qquad (2)

where: $\delta R$ — first variation under $u\to u+\epsilon\eta$ ; $\lambda$ — eigenvalue, equal to $R[u]$ at the stationary point. Reading: to find an eigenpair you may either solve a differential equation or seek a stationary point of a scalar functional. For a self-adjoint operator bounded below, the ground-state eigenvalue is the global minimum of the Rayleigh quotient, while excited-state eigenvalues are characterised by constrained minimisation or, equivalently, the Courant–Fischer min–max principle.

\lambda_1=\min_{u\neq0}R[u],\qquad \lambda_n=\min_{\substack{V\subset\mathcal H\\ \dim V=n}}\ \max_{u\in V}R[u] \qquad (3)

where: $\lambda_1\le\lambda_2\le\cdots$ — eigenvalues in increasing order; $V$ — an $n$ -dimensional trial subspace; the inner minimax is Courant–Fischer. Reading: the second form pins $\lambda_n$ without knowing a single lower eigenfunction — legitimising finite-basis estimates of excited states. For the operator governing vibrating strings, quantum wells, and waveguide modes, the regular Sturm–Liouville form makes the quotient a ratio of quadratic functionals once integrated by parts.

L_{SL}u=-\frac{\mathrm{d}}{\mathrm{d}x}\!\left(p\,\frac{\mathrm{d}u}{\mathrm{d}x}\right)+q\,u=\lambda\,w\,u,\qquad R[u]=\frac{\displaystyle\int_a^b\!\big(p\,u'^2+q\,u^2\big)\,\mathrm{d}x-\big[p\,u\,u'\big]_a^b}{\displaystyle\int_a^b w\,u^2\,\mathrm{d}x} \qquad (4)

where: $p(x)>0,\ q(x),\ w(x)>0$ — the Sturm–Liouville coefficients; $u'=\mathrm{d}u/\mathrm{d}x$ ; $[p\,u\,u']_a^b$ — the boundary term, vanishing under standard self-adjoint conditions. Reading: the numerator is a strain-energy-like quadratic form; $q\ge0,\ p>0$ make it non-negative, so the spectrum is bounded below and the minimisation well posed. Restrict $u$ to a finite span $u=\sum_j c_j\phi_j$ and stationarity in the coefficients turns the problem into linear algebra: a generalised matrix eigenproblem in the stiffness matrix $H$ and overlap matrix $S$ .

H_{ij}=\langle\phi_i,L\phi_j\rangle,\qquad S_{ij}=\langle\phi_i,\phi_j\rangle,\qquad H\mathbf{c}=\lambda\,S\,\mathbf{c} \qquad (5)

where: $\phi_i$ — chosen (not necessarily orthogonal) basis functions; $H$ — Hamiltonian or stiffness matrix; $S$ — overlap or mass matrix; $\mathbf{c}$ — expansion coefficients. Reading: $S\neq I$ because a practical basis is often non-orthogonal — why the problem is generalised, not a standard eigenproblem. The Ritz roots provide upper bounds on the corresponding exact eigenvalues; for nested trial spaces, enlarging the basis lowers each Ritz estimate, and the new roots interlace the old ones.

\lambda_n^{(N+1)}\le\lambda_n^{(N)},\qquad \lambda_n^{(N)}\le\lambda_{n+1}^{(N+1)} \qquad (6)

where: $\lambda_n^{(N)}$ — the $n$ -th Ritz value from an $N$ -function trial space; for nested trial spaces, the left inequality gives monotone convergence from above, while the right inequality is the Cauchy interlacing relation. Reading: systematic enlargement of a nested trial space only lowers the Ritz upper bounds, so the sequence of estimates provides its own convergence certificate.

Derivation AStationary points of the Rayleigh quotient are the eigenfunctions

The goal is to prove the theorem on which the entire subject rests: for admissible variations, demanding the first variation of $R[u]$ vanish is equivalent to the eigenvalue equation $Lu=\lambda u$ , with the stationary value equal to the eigenvalue itself. We vary the numerator and denominator separately, use self-adjointness together with the admissible boundary conditions to move $L$ onto the variation, and invoke the fundamental lemma to remove the arbitrary test function — establishing the equivalence between the differential eigenvalue problem and the variational formulation.

Step 1. Write $R=N/D$ with $N=\langle u,Lu\rangle$ , $D=\langle u,u\rangle$ , and perturb $u\to u+\epsilon\eta$ for arbitrary admissible $\eta$ and small real $\epsilon$ . By the quotient rule the first variation is $\delta R=(\delta N\,D-N\,\delta D)/D^2$ . Why: the $D^2$ denominator is harmless since $D>0$ for any non-zero $u$ .

Step 2. Self-adjointness $\langle\eta,Lu\rangle=\langle Lu,\eta\rangle$ gives $\delta N=2\,\mathrm{Re}\,\langle\eta,Lu\rangle$ and $\delta D=2\,\mathrm{Re}\,\langle\eta,u\rangle$ , so $\delta R=\dfrac{2}{D}\,\mathrm{Re}\,\langle\eta,\ Lu-R[u]\,u\rangle$ . Why: this is the step that needs $L=L^\dagger$ — without it the variation would not collapse to a single inner product and eigenfunctions would not be stationary.

Step 3. Demand $\delta R=0$ for every admissible $\eta$ . Taking $\eta$ and $i\eta$ in turn, the fundamental lemma forces the bracket to vanish: $Lu-R[u]\,u=0$ . Why: a vector orthogonal to every direction is the zero vector — the calculus-of-variations analogue of "gradient equals zero".

Step 4. So the stationarity condition is $Lu=R[u]\,u$ : the eigenvalue equation with $\lambda=R[u]$ . Conversely $Lu=\lambda u$ gives $R[u]=\lambda$ , so eigenfunctions are stationary. Why: the equivalence runs both ways — "stationary point of $R$ " and "eigenfunction of $L$ " are the same set.

Go deeper · Derivation B · 1 minWhy every trial function bounds the ground state — the spectral proof

Derivation BWhy every trial function bounds the ground state — the spectral proof

The goal is to turn the stationarity theorem into the practical bound variational calculations use: $R[u]\ge\lambda_1$ for every admissible $u$ , with equality only at the true ground state, and to extend it to excited states. Expanding an arbitrary trial function in the complete orthonormal eigenbasis makes the quotient a weighted average of eigenvalues, after which the bounds are elementary facts about averages — holding for any $u$ , however crude, not merely near-eigenfunctions.

Step 1. Assume $L$ self-adjoint with orthonormal eigenfunctions $u_k$ , eigenvalues $\lambda_1\le\lambda_2\le\cdots$ , and expand $u=\sum_k c_k u_k$ with $c_k=\langle u_k,u\rangle$ . Why: the eigenfunctions form a complete orthonormal set, so no generality is lost by working in coordinates aligned with the spectrum.

Step 2. Insert this into the quotient. Using $Lu_k=\lambda_k u_k$ and orthonormality, $R[u]=\dfrac{\sum_k\lambda_k|c_k|^2}{\sum_k|c_k|^2}$ — a weighted average of the eigenvalues with weights $|c_k|^2\ge0$ . Why: this rewrite is the heart of the subject; once $R$ is manifestly an average of eigenvalues, every bound is a statement about averages.

Step 3. Bound below by replacing every $\lambda_k$ with the smallest: $R[u]\ge\lambda_1$ , with equality precisely when $u$ lies entirely in the ground-state eigenspace. For a nondegenerate ground state, this reduces to $c_1\neq0$ and $c_{k>1}=0$ , i.e. $u\propto u_1$ . Thus $\lambda_1=\min_{u\ne0}R[u]$ . Why: this is the Rayleigh–Ritz bound — even an imperfect trial function gives a rigorous upper bound on the true ground-state energy.

Step 4. Climb: if $u\perp u_1,\dots,u_{n-1}$ then $c_1=\cdots=c_{n-1}=0$ , the sum starts at $k=n$ , and $R[u]\geq\lambda_n$ . Equality holds when $u$ lies in the eigenspace associated with $\lambda_n$ ; for a nondegenerate $\lambda_n$ , this means $u\propto u_n$ . Hence $\lambda_n=\min_{\substack{u\ne0\\u\perp u_1,\ldots,u_{n-1}}}R[u]$ . Why: excited states are constrained by orthogonality to the lower eigenspaces, which the Ritz formulation enforces through the corresponding constrained subspace.

Go deeper · Derivation C · 1 minThe Sturm–Liouville equation and its natural boundary conditions from stationarity

Derivation CThe Sturm–Liouville equation and its natural boundary conditions from stationarity

The goal is to show that making the Sturm–Liouville quadratic functional stationary at fixed normalisation reproduces the eigenvalue ODE and generates the correct boundary conditions without our imposing them — the clean illustration of the essential-versus-natural distinction. We enforce normalisation with a Lagrange multiplier, vary, integrate by parts once, and read the bulk and boundary pieces separately; the multiplier turns out to be the eigenvalue itself, tying this to constrained variation.

Step 1. Extremise $I[u]=\int_a^b(p\,u'^2+q\,u^2)\,\mathrm{d}x$ subject to $J[u]=\int_a^b w\,u^2\,\mathrm{d}x=1$ by forming $F[u]=I[u]-\lambda\,(J[u]-1)$ . Fixing the norm is essential because $R$ is scale-invariant. Why: the multiplier $\lambda$ enters as bookkeeping but will emerge as the physical eigenvalue — the surprise of this derivation.

Step 2. Vary and keep first-order terms: $\delta F=\int_a^b(2p\,u'\eta'+2q\,u\eta-2\lambda w\,u\eta)\,\mathrm{d}x$ , with $\eta=\delta u$ arbitrary and admissible. Why: the $u'\eta'$ term still carries a derivative on $\eta$ , which the fundamental lemma needs undifferentiated.

Step 3. Integrate that term by parts, giving $\delta F=2[p\,u'\eta]_a^b+\int_a^b 2\big[-(p\,u')'+q\,u-\lambda w\,u\big]\eta\,\mathrm{d}x$ . Why: integration by parts splits the variation into a bulk integral and a boundary term — the pieces that give the ODE and the boundary conditions.

Step 4. Since $\eta$ is arbitrary in the interior, the bracket vanishes pointwise: $-(p\,u')'+q\,u=\lambda\,w\,u$ , the Sturm–Liouville equation, with the multiplier revealed as the eigenvalue $\lambda$ . Why: this recovers the operator of equation (4) from pure extremisation — solving the ODE and minimising the quotient are the same task.

Step 5. The leftover $[p\,u'\eta]_a^b=0$ must also hold. Where $u$ is pinned (essential/Dirichlet) $\eta$ vanishes and the term dies; where $u$ is free, $\eta\neq0$ forces $p\,u'=0$ — a natural Neumann-type condition supplied for free. Why: the operational meaning of "natural versus essential" — you impose essential conditions by hand, but the natural ones follow from stationarity.

Limiting cases

The variational characterisation sharpens, specialises, and — instructively — fails in identifiable regimes.

Case	Result	Meaning
Ground state	$\lambda_1=\min_{u\neq0}\dfrac{\langle u,Lu\rangle}{\langle u,u\rangle}$	Unconstrained minimum; any admissible trial function is a rigorous upper bound on $\lambda_1$ .
Excited states, modes known	$\lambda_n=\min_{u\perp u_1,\dots,u_{n-1}}R[u]$	A constrained minimum requiring the lower eigenfunctions in advance.
Excited states, minimax	$\lambda_n=\min_{\dim V=n}\max_{u\in V}R[u]$	Courant–Fischer: fixes $\lambda_n$ over all $n$ -dimensional subspaces, needing no lower modes.
Trial function near an eigenfunction	$u=u_n+\epsilon\,\delta u\Rightarrow R=\lambda_n+O(\epsilon^2)$	Quadratic insensitivity: a $1\%$ shape error costs only $\sim0.01\%$ in the eigenvalue.
Non-self-adjoint $L$	$R[u]\notin\mathbb R$ in general	The bounding property fails; eigenvalues may be complex, and the usual Rayleigh quotient no longer provides a real minimum or upper-bound principle.

Worked examples

1 · Particle in a box, and how the bound tightens (fully worked)

Take $L=-\mathrm{d}^2/\mathrm{d}x^2$ with $w=1$ on $[0,1]$ and $u(0)=u(1)=0$ — the infinite square well, exact spectrum $\lambda_n=n^2\pi^2$ , so $\lambda_1=\pi^2=9.8696$ . Estimate $\lambda_1$ from a crude single trial, then sharpen it with a two-function Rayleigh–Ritz matrix.

Step 1. Pick the simplest polynomial satisfying the essential conditions: $\phi_1=x(1-x)$ , which vanishes at both ends. Its quotient needs $\int_0^1\phi_1'^2\,\mathrm{d}x$ and $\int_0^1\phi_1^2\,\mathrm{d}x$ , with $\phi_1'=1-2x$ .

Step 2. Evaluate: $\int_0^1(1-2x)^2\,\mathrm{d}x=\tfrac13$ and $\int_0^1 x^2(1-x)^2\,\mathrm{d}x=\tfrac1{30}$ , so $R[\phi_1]=\dfrac{1/3}{1/30}=\mathbf{10}$ . Against the exact $\lambda_1=\pi^2=9.8696$ , this crude guess is an upper bound just $\mathbf{1.3\%}$ high — from one parabola. The variational energy is stationary at the exact eigenfunction, so sufficiently small errors in the trial function produce only a second-order error in the energy.

Step 3. Enlarge the basis: add $\phi_2=x^2(1-x)^2$ , also vanishing at both ends, and build the matrices of equation (5) with $H_{ij}=\int_0^1\phi_i'\phi_j'\,\mathrm{d}x$ (boundary term vanishing after parts) and $S_{ij}=\int_0^1\phi_i\phi_j\,\mathrm{d}x$ :

H=\begin{pmatrix}\tfrac13 & \tfrac1{15}\\[2pt]\tfrac1{15} & \tfrac2{105}\end{pmatrix},\qquad S=\begin{pmatrix}\tfrac1{30} & \tfrac1{140}\\[2pt]\tfrac1{140} & \tfrac1{630}\end{pmatrix}

Step 4. Solve $\det(H-\lambda S)=0$ , a quadratic in $\lambda$ , with roots $\lambda_1^{(2)}=9.8697$ and $\lambda_2^{(2)}=102.1$ . The lower root has fallen from $10$ to $9.8697$ — now only $\mathbf{0.0015\%}$ above $\pi^2$ , a thousandfold reduction in the ground-state error from one additional basis function. The monotone-convergence inequality $\lambda_1^{(2)}\le\lambda_1^{(1)}$ is made concrete.

Step 5. The second root: both $\phi_1,\phi_2$ are symmetric about $x=\frac12$ , so the trial space contains only even-parity states $n=1,3,\ldots$ . Thus the upper Ritz root $102.1$ is a (loose) upper bound on $\lambda_3=9\pi^2=88.83$ , not on $\lambda_2=4\pi^2=39.48$ . To approximate $\lambda_2$ , the trial space must include an antisymmetric function such as $x(1-x)(1-2x)$ . The ground-state estimate improves from 10 (1.3% high) to 9.8697 (0.0015% high), while each Ritz value is an upper bound to the corresponding eigenvalue within the symmetry sector represented by the trial space.

2 · The hydrogen atom by a one-parameter trial (fully worked)

Estimate the ground-state energy of hydrogen, $H=-\tfrac12\nabla^2-\dfrac1r$ in Hartree atomic units, with the trial family $\psi(r)=e^{-\alpha r}$ , $\alpha>0$ . Minimise over $\alpha$ .

Step 1. The Rayleigh quotient here is the energy expectation $E(\alpha)=\langle\psi|H|\psi\rangle/\langle\psi|\psi\rangle$ . For the exponential trial the standard integrals give the normalised expectations $\langle T\rangle=\tfrac12\alpha^2$ and $\langle V\rangle=-\alpha$ (using $\langle 1/r\rangle=\alpha$ ).

Step 2. Assemble the energy functional: $E(\alpha)=\tfrac12\alpha^2-\alpha$ — the entire variational problem reduced to a one-dimensional curve (the second figure), whose minimum is the best estimate this family can give.

Step 3. Minimise: $\dfrac{\mathrm dE}{\mathrm d\alpha}=\alpha-1=0\Rightarrow\alpha_*=\mathbf{1}$ — ordinary calculus in one parameter, the Ritz reduction in its simplest form.

Step 4. Evaluate: $E_{\min}=\tfrac12-1=\mathbf{-0.5\ \text{Ha}=-13.6\ \text{eV}}$ — the exact hydrogen ground state, because the true eigenfunction $e^{-r}$ lies inside the family at $\alpha=1$ . When the family contains the exact eigenfunction, the minimum hits it and the bound becomes an equality.

Step 5. Contrast a family that does not contain the answer: a single Gaussian $\psi=e^{-\alpha r^2}$ , optimised the same way, gives $E_{\min}=-\dfrac{4}{3\pi}=-0.4244$ Ha — a genuine upper bound $\sim15\%$ above the truth, because a Gaussian has the wrong cusp and tail. The exponential trial saturates at −0.5 Ha; the Gaussian gives a strict bound of −0.424 Ha — the estimate equals the truth only when the exact state is reachable.

Physics of the portrait

The Rayleigh quotient records two names and spans acoustics, quantum mechanics, and numerical analysis — from a trick for guessing a bell's pitch to the organising principle of computational physics.

Origins · 1 minRayleigh quotient and how this was discovered

Rayleigh quotient

the eigenvalue functional R = ⟨u,Lu⟩ / ⟨u,u⟩ · variational spectral theory

The quotient is named for John William Strutt, third Baron Rayleigh, who in The Theory of Sound (1877) needed the fundamental frequencies of drums, bells, and plates without solving their eigenvalue equations. His insight — Rayleigh's principle — was that the square of the lowest natural frequency is the minimum of a ratio of potential to kinetic energy over all admissible shapes, so a guessed mode shape gives a frequency always too high but usually close. The decisive generalisation came from Walther Ritz, who in 1909, in one of the last papers of a life cut short by tuberculosis at thirty-one, made the guess systematic: expand the trial function in a finite basis with adjustable coefficients and minimise over them, turning the variational problem into linear algebra. The Rayleigh–Ritz method was born, and with it the template for finite elements (made explicit by Richard Courant in 1943) and basis-set quantum chemistry. The rigorous spectral theory — that these minima really are the eigenvalues, with the minimax characterisation reaching excited states — was supplied by Ernst Fischer (1905) and Courant (1920), and Egil Hylleraas turned it into a precision instrument in 1929, computing the helium ground state to spectroscopic accuracy. What Rayleigh introduced to guess a pitch now runs, as $H\mathbf c=\lambda S\mathbf c$ , inside every electronic-structure and structural-vibration code on Earth.

History

1877

— Rayleigh states his principle for the fundamental frequency in The Theory of Sound

1905

— Fischer gives the minimax characterisation of eigenvalues

1909

— Ritz publishes the direct method of minimising over adjustable trial functions

1920

— Courant recasts the minimax principle for the operators of mathematical physics

1929

— Hylleraas computes the helium ground state variationally to spectroscopic accuracy

1943

— Courant frames the piecewise-linear trial spaces that become finite elements

Going further

Texts. Stone & Goldbart, Mathematics for Physics, Ch. 1 (Calculus of Variations) and Ch. 4 (Linear Differential Operators) · Arfken, Weber & Harris, Mathematical Methods for Physicists, 7th ed., Ch. 22 · Hassani, Mathematical Physics, 2nd ed., Ch. 19 (Sturm–Liouville systems) · Byron & Fuller, Mathematics of Classical and Quantum Physics, Ch. 5 · Courant & Hilbert, Methods of Mathematical Physics, Vol. I, Ch. VI · Gould, Variational Methods for Eigenvalue Problems. Key numerical checks: the first variation of $R$ yields $Lu=\lambda u$ ; the spectral-average argument gives $\lambda_1\le R[u]\le\lambda_{\max}$ ; the box trial $x(1-x)$ gives $R=10$ (1.3% above $\pi^2$ ) and the $2\times2$ Ritz matrix gives $9.8697$ (0.0015% high) and $102.1$ ; the hydrogen trial $E(\alpha)=\alpha^2/2-\alpha$ gives $-0.5$ Ha at $\alpha=1$ , and the single-Gaussian bound is $-4/(3\pi)=-0.424$ Ha.

Prerequisites. Functionals and Function Spaces · First Variation and Stationarity · Euler–Lagrange Equation · Constrained Variation and Lagrange Multipliers · Natural and Essential Boundary Conditions. Leads to. Isoperimetric Problems · Hamilton's Principle of Stationary Action · Geodesics as Variational Problems.

Ask the tutor: Ask me to walk you through why a wrong guess still bounds the ground state, or to set up the Rayleigh–Ritz matrix for a potential of your choosing.

Is the Rayleigh quotient the same thing as the energy expectation value?

For the quantum Hamiltonian, yes — $R[\psi]=\langle\psi|H|\psi\rangle/\langle\psi|\psi\rangle$ is exactly the expectation value of the energy in the (unnormalised) state $\psi$ , and the variational principle reads as physics: the ground state is the configuration of least energy at fixed normalisation. But the quotient is more general. Replace $H$ by any self-adjoint operator — the Laplacian of a drum, the stiffness operator of a beam, the Sturm–Liouville operator of a waveguide — and $R$ becomes the relevant "energy per unit norm", whose minimum is the lowest frequency or the buckling load. The quantum reading is the most famous special case, not the definition.

Why does the eigenvalue come out so much more accurately than the wavefunction?

Because $R$ is stationary at every eigenfunction, its first variation there is zero. Write the trial function as the true eigenfunction plus a small error, $u=u_n+\epsilon\,\delta u$ ; the leading correction to $R$ is not order $\epsilon$ but order $\epsilon^2$ , since the linear term vanishes by stationarity. So a $1\%$ error in the shape produces only about $0.01\%$ error in the eigenvalue. This quadratic insensitivity is the most useful fact about variational methods: a modest, physically motivated guess delivers an energy far more accurate than the guess deserves, which is why the method dominates practical quantum chemistry.

What is the difference between the constrained-minimum and minimax routes to excited states?

Both compute $\lambda_n$ , but need different information. The constrained-minimum formula $\lambda_n=\min_{u\perp u_1,\dots,u_{n-1}}R[u]$ requires you to already know the lower eigenfunctions, so you can restrict to their orthogonal complement — practical only when the low modes are in hand. The Courant–Fischer minimax formula $\lambda_n=\min_{\dim V=n}\max_{u\in V}R[u]$ needs nothing of the kind: it ranges over all $n$ -dimensional subspaces and takes the worst-case quotient in each. The minimax route legitimises finite-basis estimates of excited states, because diagonalising the Rayleigh–Ritz matrix in an $N$ -function space realises exactly this minimax over the subspaces it can build.

Why must a trial function satisfy the essential boundary conditions but not the natural ones?

The essential (Dirichlet-type) conditions define the space of admissible functions — they are part of the problem's statement, and a function that violates them is not in the game at all. Impose them and the boundary term in the variation vanishes because the variation itself is pinned there. The natural (Neumann-type) conditions are different: they emerge from setting the boundary term to zero at a free end, so stationarity supplies them automatically. A trial function need not satisfy a natural condition in advance; if it does not, the variational principle simply enforces it in the limit. Violating an essential condition, by contrast, can push the Rayleigh quotient below the true ground state and wreck the bound.

Does the variational bound work for excited states too, or only the ground state?

The clean upper bound $\lambda_1\le R[u]$ is unconditional only for the ground state. For excited states you get a bound only if you respect orthogonality: a trial function orthogonal to the exact lower eigenfunctions bounds $\lambda_n$ from above. In practice you rarely know those exactly, so instead you diagonalise a Rayleigh–Ritz matrix in a multi-function basis — its $n$ -th root is a rigorous upper bound on $\lambda_n$ , guaranteed by the Courant–Fischer minimax principle and Cauchy interlacing. Excited-state bounds are real, but they come from the matrix method, not from naively minimising a single trial function.

When does the variational estimate become exact rather than just an upper bound?

Exactly when the trial family contains the true eigenfunction. If the exact ground state lies inside your trial space — as $e^{-r}$ lies inside the family $e^{-\alpha r}$ for hydrogen — then the minimisation finds it and the bound is saturated: $R=\lambda_1$ with no gap. This is why the hydrogen exponential trial returns $-0.5$ Ha exactly, while a Gaussian trial, whose family excludes the true cusped eigenfunction, can only reach $-0.424$ Ha. In the finite-basis picture, the estimate becomes exact when the basis spans the true eigenfunction, which is the sense in which enlarging a complete basis converges to the truth. This is also why the finite element method delivers guaranteed bounds: refining the mesh enlarges the trial space, so computed frequencies fall toward the exact ones from above.