Expectation Values and Variance
A single quantum measurement tells you almost nothing. Prepare an electron in a superposition of box states, measure its energy once, and you get one eigenvalue — a coin flip's worth of information about a state that took real work to prepare. The theory's actual predictions live one level up, in the statistics of many identically prepared systems: the mean of the outcomes, and the spread around that mean. This article builds both objects in the formalism's own language — the expectation value and the variance — and proves the identities that make them computable and trustworthy. Two derivations carry the page. The first unpacks the sandwich: inserting the eigenbasis turns into — a Born-probability-weighted average, which is why the abstract sandwich deserves the name "expectation value" — and the same move handles every moment at once. The second rewrites the variance as a squared length, : variance is geometry, manifestly non-negative, zero exactly on eigenstates, and — three articles from now — the object the generalized uncertainty principle is built from. Then the payoff modern textbooks skip: what do these numbers do? Chebyshev's inequality turns σ into a guarantee — no more than of your measurement records can land more than from the mean, whatever the spectrum looks like. And the shot-budget rule tells you how many runs buy how many digits of — the arithmetic that governs every expectation-value estimate on today's quantum processors, where "measure the energy" literally means thousands of prepared-and-collapsed circuits, averaged.
- is the ensemble mean: many copies, one measurement each — not repeated measurement of one copy (collapse forbids that reading).
- Insert the eigenbasis: and, in one stroke, — spectral weights do all statistics.
- is real for Hermitian (recalled from the Hermiticity theorems) and need not be an eigenvalue: is a legal mean over the menu .
- Norm form: , with exactly on eigenstates — variance is a squared length.
- Chebyshev: — the model-free tail guarantee σ buys you.
- Shots: the standard error of an -run estimate is ; pinning a mean to costs runs.
- Key idea: the expectation value is the Born-weighted mean , computed without diagonalizing anything as ; the variance is its spread, and both are statements about ensembles, not single shots.
The expectation value is the ensemble mean of measurement outcomes — equal to — and the variance measures the intrinsic spread of those outcomes around it.
Where this lives today
"Estimate an expectation value" is arguably the most-executed quantum computation on Earth. Variational algorithms (VQE and descendants) approximate molecular energies as sums of Pauli-string expectations, each estimated from thousands of prepared-and-collapsed circuits; the shot budget — with σ computed exactly as in this article — dictates runtime, and entire research programs (operator grouping, classical shadows) exist to shave it. Precision metrology lives on the same ledger: atomic clocks report with standard errors ground down by and by squeezing tricks that redistribute variance between conjugate observables. Even quantum-hardware benchmarking is variance bookkeeping — readout fidelity is inferred from the spread of repeated stabilizer expectations. The sandwich you learned to unpack is a production workload.
The physical picture
Run the movie honestly. A source prepares the same state , over and over. Each copy meets a measuring device once; each measurement returns one eigenvalue and collapses that copy (last article's rule — which is why one copy cannot be polled twice). After ten thousand runs you hold a histogram: eigenvalues on the horizontal axis, relative frequencies converging to the Born weights . The expectation value is that histogram's center of mass. The standard deviation is its width. Everything this article proves is a statement about that histogram, computed without building it — straight from the state vector.
Notice what the picture rules out. The mean is not a possible reading — no single measurement ever returns it, any more than a family has 2.4 children. The spread σ is not measurement error — the apparatus is perfect in this story; the width is the state's own indeterminacy, present before any detector misbehaves. And the whole construction presumes fresh copies: measure one copy twice and the second result is rigged by collapse (probability 1 of repeating the first — the repeatability theorem), telling you about the update rule, not about . The wave-mechanics chapter met these ideas in integral clothing — and the swinging-centroid examples of its Expectation Values article. What is new at the formalism register is leverage: one abstract identity per concept, valid for any observable with any spectrum, plus the statistical guarantees that make σ operationally meaningful. The wave-register article remains the place for the correspondence story; this one owns the machinery.
The sandwich, unpacked
Why should the abstract sandwich — computable in any basis, no diagonalization required — equal the experimental mean? Because the eigenbasis unpacks it into exactly the Born-weighted average, and the Born rule article (five serials back) certified those weights as outcome probabilities.
We prove and, with no extra work, for every power . The strategy is the chapter's signature move: expand the state in the eigenbasis of (existence certified by the Hermiticity theorems plus the basis articles), let act where it is diagonal, and collect. Degenerate spectra ride along free of charge if labels basis vectors rather than distinct eigenvalues.
Step 1. Expand with (the expansion theorem of Complete Orthonormal Bases — cited, not re-proved). Apply the operator where it is trivial: .
Step 2. Close the sandwich with the bra and use orthonormality :
Why this is the experimental mean: by the Born rule, and a probability-weighted sum of outcomes is the definition of an ensemble average. The two faces — basis-free sandwich, spectral average — are the same number; use whichever the problem hands you cheaply.
Step 3. Iterate for moments. Since (powers of an operator act as powers of the eigenvalue on its eigenvectors), the identical computation gives . Every statistical question about the outcome distribution — mean, variance, skew, full histogram — is encoded in the spectral weights, extracted by sandwiches.
Step 4. Bank the reality check: for Hermitian , is real — that is Theorem 3 of Hermitian Operators and Observables, recalled here rather than re-proved; equation (1) makes it visible a second way (real eigenvalues, non-negative weights). Verified: numerically on random Hermitian matrices and random states — sandwich equals spectral sum for and to machine precision; imaginary part of the sandwich .
The mean deserves one honest disclaimer before the spread: it can be a value the system never takes. For the two-level box state of the hero figure, every measurement returns or ; the mean sits in the forbidden gap. "Expectation" is a probabilist's word, not a prophecy — what you expect is the average of many shots, never necessarily any one of them.
Variance is a length
The spread around the mean is captured by the variance, defined as the mean squared deviation. Its two standard faces are one subtraction apart, but its third face is the one the rest of this chapter runs on. Center the operator — define (a legal Hermitian operator: observable minus number) — and compute:
where: the middle equality is two lines of algebra (expand the square, use linearity of ); the right one uses Hermiticity of to fold both copies into a norm: . Read it geometrically: variance is the squared length of the vector — how far the operator pushes the state when the mean is subtracted off.
Go deeper · Derivation B · 1 minThe norm form and what it settles for free
We derive equation (2)'s norm form and harvest its three immediate consequences: non-negativity, the eigenstate criterion, and the launchpad for the uncertainty principle. The strategy is pure inner-product algebra — the payoff of doing statistics inside a Hilbert space is that "spread" becomes "length", and lengths obey geometry.
Step 1. Expand the defining mean squared deviation using linearity and :
Why the cross term collapsed: for the number — expectation is linear and the mean is a scalar, not an operator.
Step 2. Fold into a norm. is Hermitian (difference of Hermitian and real multiple of identity), so one copy hops across the inner product: .
Step 3. Harvest. (i) always — a squared length, no case analysis. (ii) : zero spread exactly when is an eigenstate, the mean then being the eigenvalue. This is the determinate-state loop the wave chapter closed — Standard Deviation and Quantum Uncertainty owns the theorem at the wave register; the norm form makes it one line here, and we recall rather than re-annex it. (iii) The vector is the input to Cauchy–Schwarz when two observables compete — precisely how the Generalized Uncertainty Principle, three serials on, will manufacture . Variance was built for that argument.
Step 4. Two-outcome closed form, for the pocket. If only two eigenvalues carry weight ( and ), the variance collapses to — maximal at the fifty-fifty split, vanishing as either weight dies. Worth memorizing: half of exam statistics is this formula. Verified: symbolically (sympy) — the two-point identity exact; numerically — norm form equals on random Hermitian/state pairs to ; eigenstate variance .
What σ guarantees: Chebyshev and the shot budget
A width is only useful if it bounds something. Chebyshev's inequality — classical probability, applied here to the spectral weights — is the guarantee: for any state, any Hermitian observable, and any ,
where: the left side sums the Born weights of all eigenvalues at least from the mean. The proof is two honest lines with the machinery already on the table: every term in is non-negative, so keeping only far-out terms and replacing each by its floor gives — divide. No Gaussian assumption, no continuum limit; the bound is model-free (and correspondingly generous — real distributions usually beat it by a lot, as the worked example checks). The second guarantee prices estimation. Measuring in the laboratory means averaging collapsed outcomes; the sample mean is itself a random variable, centered on with standard error (independent identical shots — classical statistics on quantum weights). Inverting: one more decimal digit of an expectation value costs a hundredfold more measurements. That wall is the daily economics of every variational quantum algorithm and every precision spectroscopy run — and the reason squeezed and entangled probes (which beat it) are worth billions of research euros, a story flagged for later chapters.
Limiting cases
| case | result | meaning |
|---|---|---|
| an eigenstate of | , | determinate value; the only case where one shot suffices |
| two outcomes, weights | maximal spread at 50/50; vanishes at the edges | |
| , in every state | normalization restated as statistics | |
| compatible refinement measured first | unchanged | commuting projectors don't shift means — commutator article's doctrine |
| in Chebyshev | bound : vacuous | σ constrains tails, not the bulk |
| shots | sample mean at rate | the ensemble reading made quantitative |
| mixed preparation | the density-operator article's generalization, four serials on |
Worked examples
Example 1 — energy statistics in a box (confidence builder). An electron in a infinite square well () is prepared in . Find , , and check Chebyshev.
Step 1. Weights: , (sum 1 ✓). Menu: and .
Step 2. Mean by equation (1): — in the gap, not on the menu.
Step 3. Spread by the two-point formula: , so . (Cross-check via : identical, machine-verified.)
Step 4. Chebyshev audit: the deviations are and . At the exact tail is , safely under the bound ; at the exact tail is against a bound of — the inequality is honest and generous. Bonus: pinning to needs shots.
Example 2 — spin statistics on a tilted state (exam level). A spin- is prepared along an axis tilted from : . Find the full statistics and the shot budget to distinguish this state from the state.
Step 1. Weights: , .
Step 2. Mean: .
Step 3. Spread: (both eigenvalues square to the same number), so in every state and : . Elegant closed form: — spread is maximal on the equator, zero at the poles (machine-verified).
Step 4. Budget: the state has . Separating means apart at five standard errors with needs shots — a number any modern qubit readout burns in microseconds. Statistics, not single shots, is how states are told apart.
The man who bounded the tails
Origins · 1 minPafnuty Chebyshev and how this was discovered
Pafnuty Lvovich Chebyshev built the St Petersburg school that made probability a rigorous science, and his 1867 memoir On Mean Values proved the inequality bearing his name — the first theorem to extract a guarantee from a variance with no assumptions about the distribution behind it. His motivation was earthy: artillery tables, mechanism design (his linkages still steer), and a lifelong suspicion of unproven asymptotic folklore; his students Markov and Lyapunov industrialized the rigor into the modern limit theorems. Chebyshev never heard of quantum mechanics — he died a year before the electron was glimpsed — but his bound is one of the very few statements in this article that needed no quantum upgrade whatsoever: the Born rule hands each Hermitian operator a probability distribution over its spectrum, and Chebyshev's inequality applies verbatim, spectral weights in, tail guarantee out. When a quantum engineer asserts that at most a quarter of readouts can land beyond two sigma — whatever the device, whatever the state — she is citing a Russian mechanism-designer's 1867 theorem, unmodified across one and a half centuries and one complete revolution in physics.
Going further
References. Griffiths & Schroeter, Introduction to Quantum Mechanics, 3rd ed., §1.4–1.5, §3.4 · Sakurai & Napolitano, Modern Quantum Mechanics, 2nd ed., §1.4 · Shankar, Principles of Quantum Mechanics, 2nd ed., Ch. 4 · Cohen-Tannoudji, Diu & Laloë, Quantum Mechanics, Vol. I, Ch. III · Chebyshev, Journal de mathématiques pures et appliquées 12, 177 (1867).
Provenance. All equations are standard textbook results. Verifier should recompute: (1) the spectral expansion against Griffiths §3.4 / Sakurai §1.4; (2) the variance identity and its norm form; (3) reality of expectation values for Hermitian operators. For this edition all three were machine-executed (numpy/sympy) on random Hermitian matrices and states, plus this article's additions: the two-point variance identity (symbolic, exact); the box example ( eV at nm, eV, eV, Chebyshev tails at and at ); the tilted-spin example (; ; ; for -standard-error separation); Chebyshev on random discrete distributions; and the empirical standard-error check at . Per the chapter ownership map: reality of is Hermitian Operators and Observables' theorem (recalled), the eigenstate converse belongs to the wave chapter's Standard Deviation and Quantum Uncertainty (recalled by title), Born probabilities to Born Rule and Measurement Probabilities, and the mixed-state average to Density Operator and Mixed States ahead — this article owns the spectral-moments machinery, the norm form, and the statistical guarantees.
Builds on: Hermitian Operators and Observables · Born Rule and Measurement Probabilities · Inner Product and Norm of States · Completeness Relation and Resolution of Identity · Eigenvalues and Eigenstates
Leads to: Generalized Uncertainty Principle · Ehrenfest's Theorem · Measurement and Wavefunction Collapse · Density Operator and Mixed States
Is ⟨Â⟩ what I get if I measure the same particle many times?
No — that experiment measures the collapse postulate, not the state. After the first measurement the particle is in an eigenstate, and every subsequent measurement of the same observable repeats the first answer with probability 1. The expectation value is an average over fresh copies: prepare, measure once, discard, repeat. Ensemble language is not pedantry; it is the difference between statistics of and statistics of the update rule.
Why is the variance defined with squares instead of absolute deviations?
Three compounding reasons. Squares make the algebra close: expands into moments, while requires spectral surgery to even define. Squares turn variance into a norm (equation 2), inheriting all of inner-product geometry — non-negativity, the eigenstate criterion, Cauchy–Schwarz. And squares are what the uncertainty principle needs: Robertson's bound is a statement about . Mean absolute deviation is a legitimate statistic; it just buys none of this machinery.
Can two different states have identical ⟨Â⟩ and σ_A?
Easily — even identical full statistics of one observable. Any relative phase between energy eigenstate components is invisible to every : and share every energy moment forever. The phase shows up in the statistics of observables that don't commute with (position, say). One observable's histogram never determines the state; tomography — reconstructing or ρ from many observables' statistics — exists precisely because of this.
Does ⟨Â⟩ have to be a real number even in a superposition?
Yes, for any Hermitian and any state — that is Theorem 3 of the Hermiticity article, and equation (1) re-exhibits it: real eigenvalues, non-negative weights, real sum. If a calculation hands you a complex expectation value, an error upstream is guaranteed — a non-Hermitian operator slipped in (measure without symmetrizing and you'll see 's worth of complex part) or a conjugation was dropped in the sandwich.
How do I compute ⟨Â⟩ if I can't diagonalize Â?
That is the sandwich's whole point: needs matrix elements in any basis you like — — no spectrum required. Diagonalization buys the probability-distribution reading (equation 1), not the number. In practice one often goes the other way: hardware estimates by sampling, precisely because diagonalizing the operator classically is the expensive step.
Is Chebyshev ever tight, or always a loose overestimate?
It is tight exactly for the distribution that puts weight at and the rest at the mean — a legal quantum state on a three-eigenvalue observable, so no better model-free bound exists. Real spectra usually beat it enormously (Gaussian tails at : against the guaranteed ). Use Chebyshev when you know only σ; use the actual weights when you have them — as in the worked example, where the exact tail at is zero.
What replaces ⟨ψ|Â|ψ⟩ when the preparation itself is random?
The density operator: , where ρ blends quantum weights and classical preparation ignorance in one object. The formula reduces to the sandwich for a pure ρ, and the variance machinery carries over verbatim. That upgrade — and why two very different preparations can share every statistic — is the business of Density Operator and Mixed States, four serials ahead.