The Variational Principle

Every quantum chemistry software package in daily use — Gaussian, ORCA, Psi4, Q-Chem, NWChem — solves some form of the variational problem at its core. Hartree-Fock theory is a variational method restricted to single-Slater-determinant trial wavefunctions; the Roothaan-Hall equations that emerge from it are literally the stationarity condition $\partial E/\partial c_i = 0$ of the second derivation applied to molecular-orbital expansion coefficients. Configuration interaction and multi-configurational self-consistent field methods extend the trial-function space to linear combinations of determinants, again minimizing energy subject to normalization, and the resulting energies are guaranteed upper bounds to the true (non-relativistic, Born-Oppenheimer) ground-state energy — a property routinely used to benchmark and rank the accuracy of competing methods, since a lower computed energy (for a size-consistent, variational method, at fixed basis set) is provably closer to the truth. Kohn-Sham density functional theory, though formally built on a different functional (of the electron density rather than the wavefunction), inherits an analogous variational structure through the Hohenberg-Kohn theorems, and DFT total-energy minimization software uses essentially the same numerical optimization technology (self-consistent field iteration, direct inversion in the iterative subspace) originally developed for wavefunction-based variational calculations. In drug design, variational quantum chemistry underlies the calculation of binding energies and conformational energy surfaces used in computer-aided drug discovery pipelines; in materials science, variational plane-wave DFT calculations predict formation energies, band structures, and phase stability for battery materials, catalysts, and semiconductors. In quantum computing, the Variational Quantum Eigensolver (VQE) — a leading near-term algorithm for simulating molecular electronic structure on quantum hardware — is a direct, literal descendant of this theorem: a quantum computer prepares a parameterized trial state, a classical computer measures its energy and adjusts the parameters downward, exploiting exactly the guarantee proven here that lower measured energy means closer to truth.

:::quiz[For the harmonic oscillator with trial Gaussian exponent α = 0.7 mω/ħ, the text computes E[φ] to be approximately]

• 0.500 ħω
• 0.529 ħω ✓
• 0.700 ħω
• 0.250 ħω

:::quiz[The equality E[φ] = E₀ in the variational principle holds if and only if]

• φ is any normalized function
• φ is orthogonal to all excited states but not necessarily equal to ψ₀
• φ equals the true ground-state wavefunction ψ₀ (up to phase, or membership in a degenerate ground manifold) ✓
• φ is a linear combination of exactly two eigenstates

:::quiz[For the hydrogen-atom Slater trial function with ζ = 0.85, the computed E[φ] was]

• exactly -0.5 Ha
• -0.48875 Ha, higher than the true -0.5 Ha ✓
• -0.51125 Ha, lower than the true -0.5 Ha
• 0.36125 Ha

Ask the tutor: How does the Rayleigh–Ritz linear variational method turn this inequality into a matrix eigenvalue problem with one bound per root?

Picture the space of all normalizable, well-behaved functions of the electronic coordinates — an infinite-dimensional space, but one you can still picture as a landscape. Over this landscape you can define a single number for every function $\phi$ you drop into it: the energy expectation value,

This is a functional — a number attached to an entire function, not to a point — and the variational principle says this functional has a global minimum, and that minimum value is exactly $E_0$, the ground-state energy of $\hat H$. The minimizer itself — the function that achieves this floor — is $\psi_0$, the true ground-state wavefunction (or, in the degenerate case, any wavefunction in the ground-state manifold). The physical intuition is that the exact eigenstates of $\hat H$ form a complete orthonormal basis, so any trial function, however creatively you invented it, is secretly a superposition of true eigenstates $\psi_0, \psi_1, \psi_2, \dots$ with energies $E_0 \le E_1 \le E_2 \le \dots$. The expectation value of $\hat H$ in this superposition is a weighted average of the true energies, weighted by how much probability the trial function assigns to each eigenstate. A weighted average of numbers can never be smaller than the smallest number being averaged — it can only equal that smallest number if all the weight sits on it. This is the entire theorem in one sentence; the rest is bookkeeping. Chemically, this means a trial wavefunction that is missing some correct nodal structure, has the wrong cusp behavior near a nucleus, or simply lacks enough variational flexibility, will report an energy that is too high — and the deficiency is entirely traceable to "leakage" of probability into excited-state character. Improving the trial function — adding more basis functions, more variational parameters, more correlation — can only push the reported energy down toward $E_0$, never past it. This is why optimization algorithms (steepest descent on parameters, self-consistent field iteration, configuration interaction diagonalization) are safe to run to convergence: the energy curve they trace is monotonically decreasing toward a known floor, not wandering near an unknown target.

The second picture worth holding in mind is the eigenbasis-weighting story rendered as a bar chart: a trial function is a mixture of "true" stationary states, and every unit of population sitting above the ground level costs you energy proportional to the gap.

Notice that every bar sits at or above the $E_0$ line, and the weighted average of bar heights — the reported expectation value — must therefore sit at or above $E_0$ as well, touching it only when every bar except $n=0$ has zero height.

The central inequality, valid for any normalizable function $\phi$ in the domain of a Hermitian, bounded-below Hamiltonian $\hat H$:

where:

• $\phi$ — any well-behaved trial (guessed) wavefunction, not necessarily an eigenfunction of $\hat H$;
• $\hat H$ — the true, exact Hamiltonian of the system (electronic Hamiltonian under the The Born-Oppenheimer Approximation, for a molecule);
• $\langle\phi|\hat H|\phi\rangle$ — the expectation value of the Hamiltonian in the (possibly unnormalized) state $\phi$;
• $\langle\phi|\phi\rangle$ — the normalization integral, included explicitly so $\phi$ need not be pre-normalized;
• $E_0$ — the true, exact ground-state eigenvalue of $\hat H$, defined by $\hat H\psi_0 = E_0\psi_0$.

Reading: no matter how you construct $\phi$ — a clever ansatz, a linear combination of atomic orbitals, a Gaussian, a wild guess — computing its Rayleigh quotient can only report a number that is too high or exactly right. It can never accidentally report a number below the truth. This is what makes energy minimization a legitimate search strategy rather than a leap of faith. A second, equally important statement pins down the equality condition:

where:

• the forward direction is the content of the main theorem (equality is achievable);
• the reverse direction says equality is achieved only by the true ground state (up to a phase and, if the ground level is degenerate, any member of that degenerate manifold).

Reading: hitting the floor exactly is diagnostic — if your trial function's expectation value ever equals $E_0$, you have not merely approximated the ground state, you have found it exactly (or found a member of its degenerate subspace). A third relation quantifies how much excess energy you pay for using an imperfect trial function, connecting the theorem to a genuine error bound:

where:

• $c_n = \langle\psi_n|\phi\rangle$ are the (unknown, but conceptually well-defined) overlaps of the trial function with the true excited eigenstates $\psi_n$;
• $E_n - E_0 \ge 0$ are the true excitation energies;
• the sum runs over all excited states $n \ge 1$.

Reading: the "energy error" is a strictly non-negative sum, vanishing termwise only if every $c_n = 0$ for $n\ne0$ — i.e., only if $\phi$ has zero overlap with every excited state, which forces $\phi \propto \psi_0$.

Example 1 — Foundational: a deliberately wrong Gaussian guess for the harmonic oscillator. Take the quantum harmonic oscillator Hamiltonian,

whose exact ground state is $\psi_0(x) = (m\omega/\pi\hbar)^{1/4}e^{-m\omega x^2/2\hbar}$ with exact energy $E_0 = \tfrac12\hbar\omega$. To test the variational principle honestly, choose a trial function with the wrong exponent — not the optimal one — so the check is non-trivial. Let

(a Gaussian narrower than the true ground state would require if $\alpha$ were $m\omega/2\hbar$; here $\alpha$ is deliberately off by a factor of 1.4 from the true value $m\omega/2\hbar$, and this trial function is already normalized: $\int_{-\infty}^{\infty}|\phi|^2 dx = 1$). Using the standard Gaussian integrals $\int x^2 e^{-2\alpha x^2}dx = \tfrac14\sqrt{\pi/(2\alpha)^3}\cdot 2 = \frac{1}{4\alpha}\sqrt{\pi/2\alpha}$ properly normalized, the known closed form for a Gaussian trial function's energy is

Substitute $\alpha = 0.7\,m\omega/\hbar$:

Compare to the exact value $E_0 = 0.5000\,\hbar\omega$. Indeed $0.5286\,\hbar\omega > 0.5000\,\hbar\omega$, exactly as the theorem demands — the deliberately mis-tuned Gaussian overshoots by $0.0286\,\hbar\omega$, about 5.7%.

Limiting-case check. Minimize $E[\phi]$ over $\alpha$ by setting $dE/d\alpha = \hbar^2/2m - m\omega^2/8\alpha^2 = 0$, giving $\alpha_{\text{opt}} = m\omega/2\hbar$ — recovering the exact ground-state Gaussian exponent — and substituting back gives $E[\phi]_{\min} = \hbar\omega/4 + \hbar\omega/4 = \hbar\omega/2 = E_0$ exactly. This confirms that the family of Gaussian trial functions contains the true ground state as a special case, and only at that special value does the theorem's equality condition trigger.

Example 2 — Applied: the ground state of the hydrogen atom probed with a "wrong" trial exponent. Consider the hydrogen atom Hamiltonian (atomic units, $\hbar=m_e=e=4\pi\epsilon_0=1$),

with exact ground state $\psi_0 = \pi^{-1/2}e^{-r}$ and $E_0 = -0.5\text{ Ha}$. Suppose a chemist, in the spirit of Slater-type-orbital fitting, guesses an exponential trial function with the "wrong" screening parameter $\zeta = 0.85$ (representing, say, an under-screened attempt at a many-electron atom mapped naively onto hydrogen):

The general result for a normalized $1s$-type Slater trial function on a one-electron atom with nuclear charge $Z=1$ is the standard closed form

Substituting $\zeta = 0.85$:

Compare to $E_0 = -0.5\text{ Ha}$: indeed $-0.48875 > -0.5$, an overestimate of $0.01125$ Ha ($\approx 0.306$ eV), consistent with the theorem. The trial function with the wrong screening parameter is a poorer, higher-energy approximation, exactly as expected because it has non-zero overlap with excited $s$-states of the true hydrogen spectrum.

Limiting-case check. Minimizing $E[\phi] = \zeta^2/2 - \zeta$ over $\zeta$ gives $dE/d\zeta = \zeta - 1 = 0 \Rightarrow \zeta_{\text{opt}} = 1$, recovering the exact hydrogen $1s$ orbital and $E[\phi]_{\min} = 1/2 - 1 = -0.5$ Ha $= E_0$ exactly — again confirming equality is achieved only at the true wavefunction.

Example 3 — Exam/competition level: bounding the ground state of a particle in a box using a non-eigenfunction trial function, and extracting a quantitative excess-energy estimate. Consider a particle of mass $m$ in an infinite square well of width $L$, $0 \le x \le L$, with exact ground state $\psi_0(x) = \sqrt{2/L}\sin(\pi x/L)$ and $E_0 = \pi^2\hbar^2/(2mL^2)$. As a classic variational exercise, use the smooth polynomial trial function (which satisfies the correct boundary conditions $\phi(0)=\phi(L)=0$ but is not a sine function and is asymmetric enough to test the theorem properly by choosing unequal weighting):

chosen with the non-round power $1.5$ specifically so this is not the standard textbook $x(L-x)$ parabola, and so no special symmetry launders a wrong derivation.

Normalization. Compute $\langle\phi|\phi\rangle = \int_0^L x^{3}(L-x)^2\,dx$ (since $\phi^2 = x^{3}(L-x)^2$). Expand $(L-x)^2 = L^2 - 2Lx + x^2$:

Compute the bracket with common denominator 60: $\frac{15}{60} - \frac{24}{60} + \frac{10}{60} = \frac{1}{60}$. So

Kinetic energy expectation. $\hat H\phi = -\dfrac{\hbar^2}{2m}\phi''$. First derivative: $\phi = x^{1.5}L - x^{2.5}$, so

Then

Expand the integrand termwise:

Sum: $0.75L^2x - (3.75+0.75)Lx^2 + 3.75x^3 = 0.75L^2x - 4.5Lx^2 + 3.75x^3$. Integrate term by term from $0$ to $L$:

So the integral of $\phi\phi''$ is $0.375L^4 - 1.5L^4 + 0.9375L^4 = -0.1875\,L^4$. Therefore

Rayleigh quotient.

Compare to the exact answer. $E_0 = \pi^2\hbar^2/(2mL^2) = 4.9348\,\hbar^2/(mL^2)$. The trial function gives $5.625\,\hbar^2/(mL^2)$, which is indeed larger:

an overestimate of about 14%, consistent with the theorem despite the asymmetric, non-round power ($x^{1.5}$) chosen specifically to avoid any accidental exact cancellation. This is a looser trial function than the familiar $x(L-x)$ parabola (which gives the well-known tighter bound $10\hbar^2/(mL^2) \approx 2.03\,E_0$... actually the parabola gives $E[\phi]=10\hbar^2/mL^2/ \approx2.03E_0$, so numerically $10 \hbar^2/(mL^2)=10/4.9348\,E_0\approx2.03E_0$ — both trial functions overshoot, as required, with the polynomial power affecting how much they overshoot but never whether they do).

The variational principle as an energy-minimization method is inseparable from two names. Lord Rayleigh (John William Strutt) developed the underlying "Rayleigh quotient" technique in the 1870s while studying the vibrational modes of mechanical and acoustic systems — the same mathematical structure (a ratio of quadratic forms bounded by the smallest eigenvalue of a differential operator) governs vibrating strings, membranes, and, decades later, the Schrödinger equation. Walther Ritz, a Swiss theoretical physicist, generalized this into what is now the Rayleigh–Ritz method: expand the unknown function in a finite basis of known functions with adjustable coefficients, and solve for the coefficients that minimize the resulting quadratic energy functional — precisely the linear-variational strategy used throughout quantum chemistry.