Quantum Relative Entropy

The quantum relative entropy $S(\rho\|\sigma)=\mathrm{Tr}\,\rho(\log\rho-\log\sigma)$ is the single most load-bearing quantity in quantum information theory, and it wears three faces at once. Structurally it is the noncommutative lift of the classical Kullback–Leibler divergence $D(p\|q)=\sum_x p_x\log(p_x/q_x)$ : replace two probability distributions by two density operators $\rho,\sigma$ on the same Hilbert space, and accept that they need not commute. Operationally it is a rate: quantum Stein's lemma says $S(\rho\|\sigma)$ is the exponential speed at which $n$ copies of $\rho$ become perfectly distinguishable from $n$ copies of $\sigma$ , the type-II error falling like $2^{-nS(\rho\|\sigma)}$ . And architecturally it is a generating functional — von Neumann entropy, mutual information, conditional entropy, the Holevo quantity, and every relative-entropy resource monotone are read off from it by choosing the reference $\sigma$ . Two properties do almost all the work. First, $S(\rho\|\sigma)\ge 0$ with equality iff $\rho=\sigma$ (Klein's inequality), so it behaves like a separation even though it is neither symmetric nor a metric and can be $+\infty$ . Second, and far deeper, it cannot increase when both arguments pass through the same quantum channel $\mathcal{N}$ — the data-processing inequality $S(\mathcal{N}(\rho)\|\mathcal{N}(\sigma))\le S(\rho\|\sigma)$ . Physically, irreversible processing can only blur two states together, never sharpen the distinction between them. An astonishing fraction of the field — strong subadditivity of the von Neumann entropy, the Holevo bound, the second law of quantum thermodynamics, converse bounds for capacities — is a corollary of this one monotonicity statement. This article takes the definition, the derivations, and the operational meaning owned by its prerequisites and shows why relative entropy is the parent from which the rest descend.

Key takeaways

The quantum relative entropy $S(\rho\|\sigma)=\mathrm{Tr}\,\rho(\log\rho-\log\sigma)$ quantifies how distinguishable a state $\rho$ is from a reference $\sigma$ , generalizing the classical Kullback–Leibler divergence to noncommuting operators.
It is nonnegative with equality iff $\rho=\sigma$ (Klein's inequality), is finite only when $\mathrm{supp}(\rho)\subseteq\mathrm{supp}(\sigma)$ , is asymmetric, and is jointly convex — never a true metric.
Its operational meaning is quantum Stein's lemma: $S(\rho\|\sigma)$ is the optimal asymptotic error exponent for distinguishing $\rho$ from $\sigma$ by measurements on many copies.
Monotonicity under channels — the data-processing inequality — is its most powerful property: strong subadditivity, the Holevo bound, and entropy concavity all follow from it in a few lines.
Von Neumann entropy ( $\sigma=I/d$ ), mutual information ( $\sigma=\rho_A\otimes\rho_B$ ), and the Holevo $\chi$ are all special cases, making relative entropy the parent quantity of the whole theory.
Key idea: read $S(\rho\|\sigma)$ as the asymptotic statistical cost of mistaking $\rho$ for $\sigma$ — by Stein's lemma it is the optimal error exponent for many copies, and its monotonicity under channels is the engine behind strong subadditivity, the Holevo bound, and entropy concavity.

The core insight

Read $S(\rho\|\sigma)$ as the asymptotic statistical cost of mistaking $\rho$ for $\sigma$ : by Stein's lemma it is the optimal error exponent in distinguishing many copies, and its monotonicity under channels (data processing) is the engine behind strong subadditivity, the Holevo bound, and entropy concavity.

Where this lives today

The relative entropy is the working currency of quantum information, and its reach keeps widening. In quantum Shannon theory, data processing applied to $S(\rho\|\sigma)$ delivers the strong-converse capacity bounds for classical, private, and quantum communication, and it is the pivot of the "entropy method" that proves coding theorems without ad hoc tricks. In quantum thermodynamics it has become the rigorous statement of the second law at the single-system, finite-size scale: the free-energy difference $F(\rho)-F(\sigma)=k_BT\ln2\cdot S(\rho\|\sigma)$ bounds extractable work, and its monotonicity under thermal operations underlies the resource theory of athermality that governs nanoscale heat engines and Landauer erasure. In resource theories more broadly — entanglement, coherence, asymmetry, magic — the template $\min_{\sigma\in\text{free}}S(\rho\|\sigma)$ manufactures a monotone whose non-increase under free operations is guaranteed by data processing alone. And in high-energy physics and holography, relative entropy has turned out to be the sharp, ultraviolet-finite version of entanglement entropy: its positivity and monotonicity encode the quantum null energy condition and the Bekenstein bound, and the Bousso–Casini–Fisher–Maldacena program uses it to constrain quantum field theory and quantum gravity. A single 1962 definition now appears in laboratory hypothesis-testing experiments, in the design of quantum error-correcting codes, and in arguments about the interior of black holes.

The physical picture

Let $\sigma$ be the model you believe and $\rho$ be reality. The relative entropy measures, in bits, how badly your model will fail once you collect data generated by $\rho$ — how surprised you are, on average, per copy. If $\sigma$ assigns vanishing weight to something $\rho$ produces, the surprise is unbounded, which is exactly why the divergence is $+\infty$ when the supports mismatch: $\rho$ can generate an outcome that $\sigma$ swore was impossible, and one such outcome settles the question forever. The first figure turns this into the quantity a statistician actually measures — the error probability that decays as you accumulate copies, at a rate fixed by $S(\rho\|\sigma)$ .

Figure

Quantum Stein's lemma made concrete. With ρ = diag(0.75, 0.25) and σ = I/2 the relative entropy is S(ρ‖σ) = 0.1887 bits, so the optimal type-II error decays as βₙ ≈ 2 to the power minus n times 0.1887. After n = 100 copies βₙ ≈ 2×10⁻⁶ — the slope of the line is exactly the relative entropy

Now shift from the operational face to the structural one. The reason relative entropy matters so much is not that it is one distinguishability measure among many, but that nearly every named entropic quantity is $S(\rho\|\sigma)$ evaluated at a cleverly chosen reference $\sigma$ . Set $\sigma=I/d$ and you recover the von Neumann entropy as an entropy deficit; set $\sigma=\rho_A\otimes\rho_B$ and you recover the quantum mutual information; average over an ensemble and you recover the Holevo $\chi$ ; minimize over a free set of states and you build a resource monotone. The second figure lays out this hub-and-spoke structure — the single generator and the quantities that fall out of it.

Figure

Relative entropy as the generating functional of quantum information. Choosing the reference σ reproduces the named measures: σ = I/d gives the von Neumann entropy deficit, σ = ρ_A⊗ρ_B gives mutual information, an ensemble average gives the Holevo χ, a minimization over free states gives resource monotones, and a Gibbs state gives nonequilibrium free energy

Underneath both faces sits one geometric fact. Because $\log$ is operator-concave, the cross term $-\mathrm{Tr}\,\rho\log\sigma$ rewards a reference $\sigma$ for being aligned with $\rho$ , and at fixed $\rho$ the divergence is minimized — driven to zero — exactly when $\sigma=\rho$ . That is the content of Klein's inequality, and it is why the relative entropy can serve as a "distance" even though it is asymmetric and unbounded.

The core equations

The definition is Umegaki's, and its ordering matters: $\log\rho$ comes first so that the quantity is nonnegative. Everything below is either a property of this object or a computation of it.

S(\rho\|\sigma)=\mathrm{Tr}\big[\rho\,(\log\rho-\log\sigma)\big] \qquad (1)

where: $\rho,\sigma$ — density operators (positive, unit trace) on the same Hilbert space $\mathcal{H}$ ; $\log$ — operator logarithm via spectral functional calculus, base 2 (bits) or $e$ (nats); $\mathrm{Tr}$ — trace over $\mathcal{H}$ . Read it as "the expected log-likelihood ratio between $\rho$ and $\sigma$ ", taken in the noncommutative sense where the two logs need not share an eigenbasis. Positivity is the first structural pillar. It says the divergence is a genuine separation that bottoms out only at coincidence.

S(\rho\|\sigma)\;\ge\;0,\qquad S(\rho\|\sigma)=0\iff\rho=\sigma \qquad (2)

where: the equality condition is exact — any nonzero divergence certifies that the two states differ. This is Klein's inequality; Derivation 1 proves it by reducing to the classical case. Monotonicity is the second and deeper pillar. It states that no physical processing can increase the divergence between two states.

S\big(\mathcal{N}(\rho)\,\big\|\,\mathcal{N}(\sigma)\big)\;\le\;S(\rho\|\sigma) \qquad (3)

where: $\mathcal{N}$ — any completely positive trace-preserving (CPTP) quantum channel. The partial trace, a measurement, a unitary followed by discarding an ancilla — all are channels, so all obey this. Derivations 2 and 3 turn this one line into strong subadditivity and the Holevo bound. The entropy decomposition exposes the link to the von Neumann entropy and defines the cross-entropy.

S(\rho\|\sigma)=-S(\rho)-\mathrm{Tr}(\rho\log\sigma) \qquad (4)

where: $S(\rho)=-\mathrm{Tr}\,\rho\log\rho$ — von Neumann entropy; $-\mathrm{Tr}(\rho\log\sigma)$ — the quantum cross-entropy of $\rho$ relative to $\sigma$ . Setting $\sigma=I/d$ gives $S(\rho\|I/d)=\log d-S(\rho)$ , the identity that makes entropy a special case of relative entropy. The operational anchor is quantum Stein's lemma: the divergence is literally an achievable, and optimal, error rate.

\lim_{n\to\infty}-\frac{1}{n}\log\beta_n(\epsilon)=S(\rho\|\sigma) \qquad (5)

where: $\beta_n(\epsilon)$ — minimal type-II error (accept $\rho$ when the true state is $\sigma$ ) subject to type-I error $\le\epsilon$ on $n$ copies; $\epsilon\in(0,1)$ — fixed tolerance on the first error type. The remarkable content is that the answer does not depend on $\epsilon$ : the exponent is $S(\rho\|\sigma)$ for every fixed tolerance. Two named information measures are relative entropies to a product or ensemble reference.

I(A;B)_\rho = S\big(\rho_{AB}\,\big\|\,\rho_A\otimes\rho_B\big),\qquad \chi=\sum_x p_x\,S\big(\rho_x\,\big\|\,\bar\rho\big) \qquad (6)

where: $\rho_{AB}$ — joint state with marginals $\rho_A,\rho_B$ ; $\{p_x,\rho_x\}$ — an ensemble with average $\bar\rho=\sum_x p_x\rho_x$ . Mutual information is the divergence from the uncorrelated product; the Holevo $\chi$ is the average divergence of ensemble members from their mean. Finally, two analytic properties tie the divergence to trace distance and to convexity.

\tfrac12\|\rho-\sigma\|_1^2\;\le\; S(\rho\|\sigma)\,\ln 2,\qquad S\Big(\textstyle\sum_k\lambda_k\rho_k\,\big\|\,\sum_k\lambda_k\sigma_k\Big)\le\sum_k\lambda_k S(\rho_k\|\sigma_k) \qquad (7)

where: $\|\cdot\|_1$ — trace norm, with $\tfrac12\|\rho-\sigma\|_1$ the trace distance; $\{\lambda_k\}$ — any probability distribution. The left inequality is quantum Pinsker (small divergence forces operational closeness); the right is joint convexity, from which data processing and entropy concavity both descend.

Derivation AKlein's inequality and the nonnegativity of relative entropy

The goal is to prove that $S(\rho\|\sigma)\ge 0$ with equality only when $\rho=\sigma$ , starting from nothing beyond the spectral decompositions of the two states and the concavity of the logarithm. The strategy is to expand the definition in the two eigenbases, recognize that the overlaps form a doubly stochastic matrix, apply Jensen's inequality to push the concave logarithm inside an average, and thereby collapse the noncommutative quantity onto a classical relative entropy which is manifestly nonnegative. This is the archetype for the whole article: a quantum inequality proved by dominating it with its classical shadow.

Step 1. Diagonalize both states, $\rho=\sum_i p_i|i\rangle\langle i|$ and $\sigma=\sum_j q_j|j\rangle\langle j|$ , and define the overlap matrix $P_{ij}=|\langle i|j\rangle|^2$ . Because the eigenvectors of each operator form an orthonormal basis, $\sum_i P_{ij}=\sum_j P_{ij}=1$ : $P$ is doubly stochastic. Why: doubly stochastic matrices are exactly the objects Jensen's inequality is built for, and they encode the noncommutativity — if $[\rho,\sigma]=0$ the bases align and $P$ becomes a permutation.

Step 2. Insert the decompositions into the definition. Using $\langle i|\log\sigma|i\rangle=\sum_j P_{ij}\log q_j$ , the divergence becomes $S(\rho\|\sigma)=\sum_i p_i\big(\log p_i-\sum_j P_{ij}\log q_j\big)$ . Why: this isolates the only place noncommutativity can hurt us — the cross term, where $\rho$ 's eigenvectors are resolved in $\sigma$ 's eigenbasis through the weights $P_{ij}$ .

Step 3. Apply Jensen's inequality to the concave logarithm: $\sum_j P_{ij}\log q_j\le\log\big(\sum_j P_{ij}q_j\big)\equiv\log r_i$ , where $r_i=\sum_j P_{ij}q_j$ . Double stochasticity guarantees $\sum_i r_i=\sum_j q_j=1$ , so $r$ is a valid probability distribution. Why: concavity lets us replace the awkward average-of-logs by the log-of-average, upgrading a lower bound on the cross term into a clean classical object.

Step 4. Substitute to obtain $S(\rho\|\sigma)\ge\sum_i p_i\log(p_i/r_i)=D(p\|r)\ge 0$ , since the classical relative entropy of two distributions is nonnegative. Equality in Jensen forces $\sigma$ 's eigenvalues to be constant across $\rho$ 's support and $P$ to be a permutation, which together force $\rho=\sigma$ . Why: the quantum divergence is squeezed between zero and a classical divergence, so it inherits both nonnegativity and the sharp equality condition. *Verified: reduces to the classical Gibbs inequality when $[\rho,\sigma]=0$ ; sign convention $\mathrm{Tr}\,\rho(\log\rho-\log\sigma)\ge0$ matches Nielsen & Chuang §11.3 and Wilde Ch. 11.*

Go deeper · Derivation B · 1 minStrong subadditivity from data processing

Derivation BStrong subadditivity from data processing

The goal is to derive strong subadditivity of the von Neumann entropy — the deepest inequality in quantum information, $S(\rho_{ABC})+S(\rho_B)\le S(\rho_{AB})+S(\rho_{BC})$ — as a one-line consequence of the monotonicity of relative entropy under a channel. The strategy is to write a mutual information across the cut $A:BC$ as a relative entropy, recognize the partial trace over $C$ as a bona fide CPTP channel, apply data processing, and then translate the resulting inequality between two mutual informations back into entropies. This is the canonical demonstration that monotonicity is the true engine and SSA is a corollary.

Step 1. Write the mutual information across the cut $A:BC$ as a relative entropy, $I(A;BC)=S(\rho_{ABC}\|\rho_A\otimes\rho_{BC})$ . Why: casting a correlation measure as a divergence from the uncorrelated product is what lets the general-purpose monotonicity theorem act on it.

Step 2. Choose the channel to be the partial trace over $C$ , $\mathcal{N}=\mathrm{Tr}_C$ . It is completely positive and trace preserving, so data processing applies to both of its arguments simultaneously. Why: discarding a subsystem is the most physical channel there is, and it is precisely the operation that turns the $A:BC$ cut into the $A:B$ cut.

Step 3. Apply monotonicity. Tracing out $C$ sends $\rho_{ABC}\mapsto\rho_{AB}$ and $\rho_A\otimes\rho_{BC}\mapsto\rho_A\otimes\rho_B$ , so $S(\rho_{AB}\|\rho_A\otimes\rho_B)\le S(\rho_{ABC}\|\rho_A\otimes\rho_{BC})$ , i.e. $I(A;B)\le I(A;BC)$ . Why: this is the intuitive statement that throwing away system $C$ cannot increase how much $A$ knows — correlations only shrink under discarding.

Step 4. Expand both mutual informations in entropies: $S(\rho_A)+S(\rho_B)-S(\rho_{AB})\le S(\rho_A)+S(\rho_{BC})-S(\rho_{ABC})$ . Cancel $S(\rho_A)$ and rearrange to reach $S(\rho_{ABC})+S(\rho_B)\le S(\rho_{AB})+S(\rho_{BC})$ . Why: the $S(\rho_A)$ terms are spectators; once they drop out, the monotonicity of mutual information is literally strong subadditivity. *Verified: equality holds iff $\rho_{ABC}$ is a quantum Markov chain $A-B-C$ (Hayden–Jozsa–Petz–Winter); consistent with Nielsen & Chuang §11.4 and Wilde Ch. 11.*

Go deeper · Derivation C · 1 minThe Holevo bound from monotonicity

Derivation CThe Holevo bound from monotonicity

The goal is to prove the Holevo bound, $I(X;Y)\le\chi=S(\bar\rho)-\sum_x p_x S(\rho_x)$ , which caps the classical information any measurement can extract about which state of an ensemble was prepared. The strategy is to package the ensemble into a classical–quantum state, recognize its mutual information as the Holevo quantity written as a relative entropy, model the receiver's measurement as a CPTP channel on the quantum register, and apply data processing so that the accessible information $I(X;Y)$ is bounded by the pre-measurement $\chi$ . It is the reason $\chi$ sets the capacity of a classical-quantum channel.

Step 1. Encode the ensemble in a classical–quantum state $\rho_{XB}=\sum_x p_x\,|x\rangle\langle x|_X\otimes(\rho_x)_B$ , where the orthonormal register $X$ records the label and $B$ carries the corresponding quantum state. Why: promoting the classical index to a quantum register lets a single mutual information hold both the "which state" information and the states themselves.

Step 2. Compute its mutual information as a relative entropy: $I(X;B)_\rho=S(\rho_{XB}\|\rho_X\otimes\rho_B)=S(\bar\rho)-\sum_x p_x S(\rho_x)=\chi$ . Why: the orthogonality of the $|x\rangle$ makes the joint entropy split cleanly, and what survives is exactly the Holevo quantity.

Step 3. Model the receiver as a measurement channel $\mathcal{M}:\sigma_B\mapsto\sum_y\mathrm{Tr}(E_y\sigma_B)\,|y\rangle\langle y|_Y$ built from a POVM $\{E_y\}$ . This is CPTP: it maps the quantum register $B$ to a classical outcome register $Y$ . Why: any physically realizable readout, however clever, is some POVM, so bounding all POVMs at once is what makes the Holevo bound universal.

Step 4. Apply data processing under $\mathrm{id}_X\otimes\mathcal{M}$ . It sends $\rho_{XB}$ to a classical–classical state whose mutual information is the accessible information $I(X;Y)$ , and monotonicity gives $I(X;Y)\le I(X;B)=\chi$ . Why: measuring is processing, and processing cannot create correlation, so the classical information you read out is capped by the quantum $\chi$ you started with. *Verified: matches Nielsen & Chuang Theorem 12.1; the bound is saturable only when the $\rho_x$ commute (are simultaneously diagonalizable).*

Limiting cases

Case	Result	Meaning
Commuting states	$[\rho,\sigma]=0\Rightarrow S(\rho\\|\sigma)=\sum_i p_i\log\tfrac{p_i}{q_i}$	Shared eigenbasis collapses the quantum quantity onto the classical Kullback–Leibler divergence $D(p\\|q)$ .
Reference $=$ maximally mixed	$S(\rho\\|I/d)=\log d-S(\rho)$	The divergence becomes the entropy deficit; this is how von Neumann entropy is generated from relative entropy.
Orthogonal / leaking supports	$\mathrm{supp}(\rho)\not\subseteq\mathrm{supp}(\sigma)\Rightarrow S(\rho\\|\sigma)=+\infty$	$\rho$ can produce an outcome $\sigma$ forbids, so a single copy can distinguish them perfectly.
Near-coincident states	$S(\rho\\|\rho+\epsilon\Delta)=\tfrac12\epsilon^2\langle\Delta,\Delta\rangle_\rho+O(\epsilon^3)$	To second order it is a quadratic form — the Kubo–Mori metric — so infinitesimally it is a Riemannian distance squared.
Pure $\rho=\|\psi\rangle\langle\psi\|$	$S(\rho\\|\sigma)=-\langle\psi\|\log\sigma\|\psi\rangle$	The $\rho\log\rho$ term vanishes ( $0\log0=0$ ), leaving pure cross-entropy against $\sigma$ .

Common trapTreating relative entropy as a symmetric distance or metric

The relative entropy is NOT symmetric — in general $S(\rho\|\sigma)\ne S(\sigma\|\rho)$ — and it is NOT a metric: it violates the triangle inequality and can be $+\infty$ . The two arguments play genuinely different roles: $\rho$ is the "true" state generating data and $\sigma$ is the reference, so swapping them changes the question. It diverges whenever $\rho$ has support outside that of $\sigma$ (an outcome $\rho$ deems possible but $\sigma$ forbids), so always check $\mathrm{supp}(\rho)\subseteq\mathrm{supp}(\sigma)$ before manipulating it. And mind the sign convention — $\log\rho$ appears first, $S(\rho\|\sigma)=\mathrm{Tr}\,\rho(\log\rho-\log\sigma)$ — which is exactly what makes the quantity nonnegative; the reversed sign would make it nonpositive and destroy every downstream theorem.

Worked examples

1 · A qubit divergence, its asymmetry, and the Stein rate (fully worked)

Let $\rho=\mathrm{diag}(0.75,0.25)$ and $\sigma=I/2=\mathrm{diag}(0.5,0.5)$ in the same basis. Compute $S(\rho\|\sigma)$ in bits, verify it via the entropy decomposition, show it differs from $S(\sigma\|\rho)$ , and read off how fast $\rho$ and $\sigma$ become distinguishable.

Step 1. The states commute, so the quantum divergence is the classical KL sum: $S(\rho\|\sigma)=0.75\log_2\tfrac{0.75}{0.5}+0.25\log_2\tfrac{0.25}{0.5}=0.75\log_2 1.5+0.25\log_2 0.5=0.75(0.58496)-0.25=\mathbf{0.1887\ \text{bits}}$ .

Step 2. Cross-check with the decomposition $S(\rho\|I/2)=\log_2 2-S(\rho)$ . The von Neumann entropy is $S(\rho)=-0.75\log_2 0.75-0.25\log_2 0.25=0.31128+0.5=0.81128$ bits, so $\log_2 2-S(\rho)=1-0.81128=\mathbf{0.1887\ \text{bits}}$ — identical, as it must be.

Step 3. Now reverse the arguments: $S(\sigma\|\rho)=0.5\log_2\tfrac{0.5}{0.75}+0.5\log_2\tfrac{0.5}{0.25}=0.5\log_2\tfrac23+0.5\log_2 2=-0.29248+0.5=\mathbf{0.2075\ \text{bits}}$ . Since $0.2075\ne 0.1887$ , the two orderings give different numbers — a concrete demonstration that relative entropy is asymmetric.

Step 4. Operationally, Stein's lemma says the optimal type-II error after $n$ copies falls as $\beta_n\approx 2^{-nS(\rho\|\sigma)}$ . For $n=100$ , $\beta_{100}\approx 2^{-18.87}\approx\mathbf{2\times10^{-6}}$ : a hundred copies already pin down which state you hold to a part in a million, and the rate of that collapse is precisely the $0.1887$ bits computed above.

2 · A noncommuting divergence you cannot do on the diagonal

Step 1. Because $\rho$ is pure, the $\rho\log\rho$ term vanishes ( $0\log 0=0$ ), and by the pure-state limiting case $S(\rho\|\sigma)=-\langle 0|\log\sigma|0\rangle$ . The whole answer is a cross-entropy — but it must be evaluated with the operator logarithm of $\sigma$ , not with $\sigma$ 's diagonal in the $Z$ basis.

Step 3. Therefore $\langle 0|\log\sigma|0\rangle=\tfrac12\log_2 0.85+\tfrac12\log_2 0.15=\tfrac12(-0.23447)+\tfrac12(-2.73697)=-1.48572$ , giving $S(\rho\|\sigma)=\mathbf{1.486\ \text{bits}}$ .

Step 4. The lesson: had you carelessly used $\sigma$ 's eigenvalues as if $\rho$ were diagonal in the $X$ basis, or used $\rho$ 's populations against $\sigma$ 's $Z$ -diagonal, you would get the wrong number. Noncommutativity lives entirely in the overlaps $|\langle 0|\pm\rangle|^2$ , and honoring it is the whole content of the quantum definition. The correct value is 1.486 bits.

3 · Relative entropy as distance from thermal equilibrium

A qubit has Hamiltonian $H=\varepsilon\,|1\rangle\langle1|$ at inverse temperature $\beta$ with $\beta\varepsilon=1$ , so the Gibbs state is $\sigma=e^{-\beta H}/Z$ with $Z=1+e^{-1}$ . Take the actual state to be maximally mixed, $\rho=I/2$ . How far from equilibrium is it, in free-energy terms?

Step 1. The Gibbs populations are $q_0=1/Z=1/1.36788=0.7311$ (ground) and $q_1=e^{-1}/Z=0.2689$ (excited). Both states are diagonal in the energy basis, so again the divergence is a classical KL sum.

Step 2. $S(\rho\|\sigma)=0.5\log_2\tfrac{0.5}{0.7311}+0.5\log_2\tfrac{0.5}{0.2689}=0.5(-0.54807)+0.5(0.89469)=\mathbf{0.1733\ \text{bits}}$ .

Step 3. The nonequilibrium free-energy difference is $F(\rho)-F(\sigma)=k_BT\ln2\cdot S(\rho\|\sigma)=k_BT\,(0.6931)(0.1733)=\mathbf{0.120\ k_BT}$ . This is the maximum work extractable by relaxing $\rho$ to equilibrium with an ideal isothermal engine, and because relative entropy is monotone under Gibbs-preserving (thermal) channels, it can only decrease as the qubit thermalizes — the second law in one inequality. The state sits 0.120 k_BT above equilibrium.

Physics of the portrait

Relative entropy is one of those quantities whose classical form was discovered by statisticians and whose quantum form waited for an operator algebraist to write it down correctly.

Origins · 1 minHisaharu Umegaki and how this was discovered

Hisaharu Umegaki

1925–2009 · Japanese mathematician · operator algebras and quantum information

Hisaharu Umegaki was a functional analyst at the Tokyo Institute of Technology who spent the late 1950s and early 1960s building the theory of conditional expectations on von Neumann algebras — the noncommutative generalization of taking a conditional average. In a 1962 paper in that series, almost as a technical tool rather than a headline result, he wrote down $S(\rho\|\sigma)=\mathrm{Tr}\,\rho(\log\rho-\log\sigma)$ and proved its basic monotonicity properties. It was the correct quantum lift of the Kullback–Leibler divergence that Solomon Kullback and Richard Leibler had introduced classically in 1951, and Umegaki's ordering of the logarithms is exactly what guarantees nonnegativity. For a decade the definition sat mostly among mathematicians. Then in 1973 Elliott Lieb and Mary Beth Ruskai proved strong subadditivity and the joint convexity of relative entropy, and in 1975 Göran Lindblad proved full monotonicity under CPTP maps — and Umegaki's formula suddenly became the hinge on which quantum information theory turned. Everything from the Holevo bound to the second law of quantum thermodynamics now routes through the object he defined. It is a recurring pattern in the field: the physically decisive quantity was first written down as a lemma in a paper about operator algebras, and only later recognized as the thing itself.

History

1951

— Kullback and Leibler define the classical relative entropy (KL divergence)

1962

— Umegaki writes down the quantum relative entropy Tr ρ(log ρ − log σ)

1973

— Lieb and Ruskai prove strong subadditivity and joint convexity

1975

— Lindblad proves monotonicity under CPTP maps (data processing)

1991

— Hiai and Petz establish the quantum Stein's lemma error exponent

Going further

Texts. Nielsen & Chuang, Quantum Computation and Quantum Information, §11.3–12.1 · Wilde, Quantum Information Theory, 2nd ed., Ch. 11 & 13 · Watrous, The Theory of Quantum Information, Ch. 5 · Preskill, Lecture Notes for Physics 219, Ch. 10 · Petz, Quantum Information Theory and Quantum Statistics · Hiai & Petz, Comm. Math. Phys. 143, 99 (1991) — quantum Stein's lemma.

Provenance. The Umegaki definition and the four structural results (Klein's inequality, strong subadditivity from data processing, concavity from joint convexity, the Holevo bound) follow Nielsen & Chuang Ch. 11–12, Wilde Ch. 11 & 13, and Watrous Ch. 5. A verifier should reconfirm: (1) the sign convention $S(\rho\|\sigma)=\mathrm{Tr}\,\rho(\log\rho-\log\sigma)\ge0$ ; (2) the Stein exponent equals $S(\rho\|\sigma)$ (Hiai–Petz 1991, Ogawa–Nagaoka strong converse); (3) the identities $S(\rho\|I/d)=\log d-S(\rho)$ and $I(A;B)=S(\rho_{AB}\|\rho_A\otimes\rho_B)$ ; (4) the worked numbers $S(\mathrm{diag}(0.75,0.25)\|I/2)=0.1887$ bits, its reverse $0.2075$ bits, the noncommuting value $1.486$ bits, and the thermal $0.120\,k_BT$ ; (5) the Pinsker constant $\tfrac12\|\rho-\sigma\|_1^2\le S(\rho\|\sigma)\ln2$ .

Prerequisites. Von Neumann Entropy · Shannon Entropy and Classical Information Measures · Quantum Channels and CPTP Maps. Leads to. Data-Processing Inequality · Strong Subadditivity of Quantum Entropy · Holevo Bound · Quantum Conditional Entropy and Mutual Information.

Ask the tutor: Ask me to walk through why data processing alone implies strong subadditivity, or to compute $S(\rho\|\sigma)$ for any pair of qubit states you name.

Why is the relative entropy asymmetric when it feels like it should be a distance?

Because the two arguments answer different questions. $S(\rho\|\sigma)$ is the statistical cost of holding $\sigma$ as your model when the data actually come from $\rho$ — it weights the log-likelihood ratio by $\rho$ , the true state. Swap them and you weight by $\sigma$ instead, asking a different question, so the numbers differ, as Example 1 shows ( $0.1887$ versus $0.2075$ bits). The asymmetry is not a defect: it reflects that mistaking a sharp distribution for a broad one is not equally costly as the reverse. When you genuinely need a symmetric, metric distance, use the trace distance or the Bures/fidelity metric; relative entropy trades the metric axioms for its unmatched operational and monotonicity properties.

What exactly does the data-processing inequality say, and why is it "the whole theory"?

It says that for any physical channel $\mathcal{N}$ , $S(\mathcal{N}(\rho)\|\mathcal{N}(\sigma))\le S(\rho\|\sigma)$ : processing two states through the same noisy evolution can only make them harder to tell apart. It is "the whole theory" in the sense that a startling number of separate-looking results are the same inequality with different channels plugged in. Choose the partial trace and you get strong subadditivity; choose a measurement and you get the Holevo bound; choose a Gibbs-preserving map and you get the second law; choose the encode-transmit-decode chain and you get converse bounds on capacity. Once you internalize data processing, these stop being independent theorems and become one principle viewed from different angles.

How is quantum relative entropy actually an error rate?

Through quantum Stein's lemma. Suppose you are handed $n$ copies of an unknown state that is either $\rho$ or $\sigma$ and must decide which, using the best possible measurement. Fix a tolerance $\epsilon$ on the probability of wrongly rejecting $\rho$ (type-I error). Then the probability of the other mistake — accepting $\rho$ when the truth was $\sigma$ (type-II error) — can be driven down to $\beta_n\approx 2^{-nS(\rho\|\sigma)}$ , and no strategy does better asymptotically. So $S(\rho\|\sigma)$ is literally the exponential rate at which the two states become distinguishable per copy. Remarkably, the rate does not depend on the tolerance $\epsilon$ you fixed, which is what makes the exponent an intrinsic property of the pair of states.

Why does the divergence blow up to infinity, and is that a problem?

It blows up precisely when $\rho$ has support where $\sigma$ has none — when $\rho$ can produce an outcome that $\sigma$ assigns probability zero. A single such outcome tells you with certainty that the state was $\rho$ , not $\sigma$ , so the states are perfectly distinguishable and the "cost of confusing them" is infinite. Formally the cross term $-\mathrm{Tr}(\rho\log\sigma)$ contains $\log 0=-\infty$ . This is not a pathology but a feature: it is the divergence correctly reporting that no amount of data is needed to separate the states. In practice it means you must check the support condition $\mathrm{supp}(\rho)\subseteq\mathrm{supp}(\sigma)$ before any algebra, and it is why regularized or smoothed versions are used when finite answers are required.

How does relative entropy generate the von Neumann entropy and mutual information?

By choosing the reference. Put $\sigma=I/d$ , the maximally mixed state, and the definition gives $S(\rho\|I/d)=\log d-S(\rho)$ , so the von Neumann entropy is $\log d$ minus the divergence from the "most ignorant" reference — entropy is an entropy deficit. Put $\sigma=\rho_A\otimes\rho_B$ , the product of the marginals, and you get $I(A;B)=S(\rho_{AB}\|\rho_A\otimes\rho_B)$ : mutual information is the divergence of the true joint state from the fictional uncorrelated one, i.e. how far the parts are from independent. Average the divergence of ensemble members from their mean and you get the Holevo $\chi$ . This is why relative entropy is called the parent quantity: the others are it, evaluated at a chosen $\sigma$ .

Is quantum relative entropy the same as the Rényi or sandwiched divergences I keep seeing?

No — the Umegaki relative entropy of this article is the $\alpha\to1$ member of a whole family. The quantum Rényi divergences interpolate a parameter $\alpha$ , and there are inequivalent quantum versions (Petz and "sandwiched") because operators do not commute, so there is no unique way to order $\rho^\alpha$ and $\sigma^{1-\alpha}$ . The sandwiched Rényi divergence governs strong-converse exponents, while the Petz version governs error-and-strong-converse tradeoffs; both reduce to Umegaki's $S(\rho\|\sigma)$ at $\alpha=1$ , which is the one Stein's lemma singles out for the fixed-tolerance asymptotic rate. For a first course, Umegaki's is the definition you want; the Rényi families become essential in refined, finite-block-length, and second-order analyses.