Quantum Entropy and von Neumann Entropy

A quantum state described by a density matrix $\rho$ on a Hilbert space of dimension $d$ is diagonalizable, since $\rho$ is Hermitian and positive semidefinite with $\mathrm{Tr}(\rho)=1$. Write its spectral decomposition as

where $\{|\phi_i\rangle\}$ is an orthonormal eigenbasis and $\lambda_i \ge 0$ satisfy $\sum_i \lambda_i = 1$. The vector $(\lambda_1,\dots,\lambda_d)$ is a genuine classical probability distribution — it describes the probabilities of outcomes if $\rho$ were measured in its own eigenbasis. The von Neumann entropy of $\rho$ is defined as the Shannon entropy of this eigenvalue distribution:

The logarithm base fixes units: base 2 gives entropy in bits (or, in the quantum context, "ebits" when discussing entanglement), while base $e$ gives nats. By convention $0\log 0 = 0$, matching the limit, so zero-probability eigenvalues contribute nothing. Two limiting cases anchor intuition. If $\rho = |\psi\rangle\langle\psi|$ is a pure state, one eigenvalue is $1$ and the rest are $0$, giving $S(\rho)=0$: a pure state carries no entropy, because it represents complete knowledge of the system. If $\rho = I_d/d$, the maximally mixed state, all $d$ eigenvalues equal $1/d$, giving $S(\rho) = \log d$, the maximum possible value for a $d$-dimensional system — total ignorance about which basis state the system occupies. The deepest use of von Neumann entropy is as an entanglement measure. Given a pure bipartite state $|\psi\rangle_{AB}$ on $\mathcal{H}_A \otimes \mathcal{H}_B$, define the reduced density matrix $\rho_A = \mathrm{Tr}_B(|\psi\rangle\langle\psi|)$ by tracing out subsystem $B$. The quantity $S(\rho_A)$ — called the entanglement entropy — is zero exactly when $|\psi\rangle_{AB}$ is a product state, and it is positive whenever $A$ and $B$ are entangled, growing to $\log(\min(d_A,d_B))$ for maximally entangled states. This is possible only because a pure global state can produce a genuinely mixed local description — a fact with no classical analogue, since a classical joint distribution that is deterministic (zero entropy) forces every marginal to also be deterministic.

Computing von Neumann entropy is an eigenvalue problem, and computing entanglement entropy adds one extra step: the partial trace. The algorithm below takes a density matrix (or a pure bipartite state vector) and returns its entropy.

Step 1 — Obtain the density matrix. If the input is a pure state $|\psi\rangle$, form $\rho = |\psi\rangle\langle\psi|$. If the input is bipartite and you want entanglement entropy, form the full density matrix on $A\otimes B$ and compute the partial trace over $B$:

Step 2 — Diagonalize. Since $\rho$ (or $\rho_A$) is Hermitian positive semidefinite with unit trace, compute its eigenvalues $\lambda_1,\dots,\lambda_d \ge 0$, $\sum_i \lambda_i = 1$.

Step 3 — Sum the classical Shannon entropy of the eigenvalue spectrum, skipping zero eigenvalues, using the appropriate log base. ``` FUNCTION VonNeumannEntropy(rho, base=2):

# rho is a Hermitian, PSD, trace-1 matrix (dxd) eigenvalues = EigenDecompose(rho) # returns real λ_i ≥ 0, Σλ_i = 1 entropy = 0 FOR each λ in eigenvalues: IF λ > 1e-12: # numerical guard for 0 log 0 = 0 entropy = entropy - λ * Log(λ, base) RETURN entropy

FUNCTION EntanglementEntropy(psi_AB, dim_A, dim_B):

# psi_AB: state vector of length dim_A dim_B, bipartition A|B rho_full = OuterProduct(psi_AB, Conjugate(psi_AB)) # (dA dB) x (dA dB) rho_A = PartialTrace(rho_full, trace_out = "B", dim_A = dim_A, dim_B = dim_B) # dA x dA RETURN VonNeumannEntropy(rho_A) ``` The `PartialTrace` routine reshapes indices $(a,b)$ into a $d_A \times d_B$ block structure and contracts the $B$ index with itself: ``` FUNCTION PartialTrace(rho, trace_out, dim_A, dim_B): result = ZeroMatrix(dim_A, dim_A) FOR a1 in 0..dim_A-1: FOR a2 in 0..dim_A-1: s = 0 FOR b in 0..dim_B-1: row = a1dim_B + b col = a2dim_B + b s = s + rho[row][col] result[a1][a2] = s RETURN result ``` A useful check: because $\rho_A$ and $\rho_B$ (traced the other way) have identical nonzero eigenvalue spectra* for any pure global state $|\psi\rangle_{AB}$, `EntanglementEntropy` gives the same answer regardless of which subsystem you trace out. This follows from the Schmidt decomposition, discussed next. The figure below shows the pipeline: a bipartite pure state's full density matrix is partially traced to a reduced density matrix, whose eigenvalue spectrum feeds the Shannon-entropy formula.

Computational complexity. For a $d$-dimensional density matrix, the dominant cost is diagonalization: standard dense eigenvalue algorithms (e.g., QR iteration after Householder tridiagonalization for a Hermitian matrix) run in $O(d^3)$ time and $O(d^2)$ space. Since $\rho$ is Hermitian, this uses the efficient real symmetric/Hermitian eigensolver rather than general complex eigenvalue routines, which would otherwise cost more due to non-normality concerns. Once eigenvalues are known, computing the entropy sum is $O(d)$, negligible next to diagonalization. For a bipartite pure state of $n$ total qubits split as $A$ ($k$ qubits) and $B$ ($n-k$ qubits), the reduced density matrix has dimension $d_A = 2^k$, so entanglement entropy costs $O(2^{3k})$ — exponential in the smaller partition's qubit count, which is unavoidable since the state itself has $2^n$ amplitudes and $\rho_A$ has $4^k$ entries to even construct via the partial trace, an $O(2^n \cdot 2^k)$ operation dominating for balanced partitions.

Correctness of the definition — why it is well-posed. $S(\rho)$ is basis-independent by construction: it depends only on eigenvalues, which are invariant under the unitary change of basis that diagonalizes $\rho$. It is real and non-negative because eigenvalues $\lambda_i \in [0,1]$ give $-\lambda_i \log \lambda_i \ge 0$ termwise. It vanishes iff exactly one eigenvalue is $1$ (all others $0$), i.e., iff $\rho$ is a rank-1 projector — precisely the definition of a pure state. It is maximized at $\log d$ by concavity of $-x\log x$ and the constraint $\sum \lambda_i = 1$: Lagrange multipliers (or Jensen's inequality applied to the concave function $-x\log x$) show the uniform distribution $\lambda_i = 1/d$ maximizes the sum, giving $S = \log d$.

Why entanglement entropy correctly measures entanglement. This rests on the Schmidt decomposition: any pure state $|\psi\rangle_{AB}$ can be written as

where $\{|a_i\rangle\}$, $\{|b_i\rangle\}$ are orthonormal sets in $A$, $B$ respectively, $p_i > 0$ sum to 1, and $r \le \min(d_A, d_B)$ is the Schmidt rank. Tracing out $B$ gives $\rho_A = \sum_i p_i |a_i\rangle\langle a_i|$ and tracing out $A$ gives $\rho_B = \sum_i p_i |b_i\rangle\langle b_i|$ — identical eigenvalue spectra $\{p_i\}$. Hence $S(\rho_A) = S(\rho_B)$ always, for any bipartite pure state, which is why entanglement entropy is unambiguous (independent of "which side" is measured). Moreover $S(\rho_A) = 0$ exactly when the Schmidt rank $r = 1$, i.e., $|\psi\rangle_{AB} = |a_1\rangle \otimes |b_1\rangle$ is a product state — no entanglement. And $S(\rho_A)$ is maximized at $\log(\min(d_A,d_B))$ exactly when all $p_i$ are equal over the maximal Schmidt rank, which is the definition of a maximally entangled state (e.g., Bell states for two qubits, giving $S = \log 2 = 1$ bit).

Key structural properties, with the reasoning behind each:

• Concavity: $S\!\left(\sum_i p_i \rho_i\right) \ge \sum_i p_i\, S(\rho_i)$. Mixing states never decreases entropy on average, because $-x\log x$ is concave and mixing density matrices can only average or increase uncertainty about which state you have — you have thrown away the label $i$ itself.
• Subadditivity: $S(\rho_{AB}) \le S(\rho_A) + S(\rho_B)$, with equality iff $\rho_{AB} = \rho_A \otimes \rho_B$ (no correlations, classical or quantum). This mirrors the classical fact that joint entropy is at most the sum of marginal entropies.
• The quantum surprise — negative conditional entropy: define $S(A|B) = S(\rho_{AB}) - S(\rho_B)$. Classically this must be $\ge 0$. Quantumly it can be negative: for a maximally entangled pure state $|\psi\rangle_{AB}$, $S(\rho_{AB}) = 0$ (globally pure) while $S(\rho_B) = \log d_B > 0$ (locally maximally mixed), giving $S(A|B) < 0$. This is the sharpest formal signature that "the whole knows less uncertainty than its part," a phenomenon impossible for any classical joint distribution and a direct quantitative witness of entanglement.
• Strong subadditivity: for three subsystems, $S(\rho_{ABC}) + S(\rho_B) \le S(\rho_{AB}) + S(\rho_{BC})$. This is the deepest known inequality for von Neumann entropy, proved via the operator-convexity of certain trace functionals (Lieb–Ruskai, 1973); it underlies the monotonicity of quantum mutual information and modern results in AdS/CFT entanglement entropy.
• Araki–Lieb inequality: $|S(\rho_A) - S(\rho_B)| \le S(\rho_{AB})$, the quantum analogue (with a twist) of the triangle inequality, tight precisely because $S(\rho_{AB})$ can be zero while $S(\rho_A) = S(\rho_B) > 0$.
• Invariance under local unitaries: $S(\rho_A)$ is unchanged if we act with any unitary $U_A \otimes U_B$ on the global pure state, since $\rho_A \to U_A \rho_A U_A^\dagger$ has the same eigenvalues. This confirms entanglement entropy depends only on the "shape" of correlations, not on local basis choices — consistent with entanglement being a resource invariant under local operations.

Operational meaning — why entropy is not just a formal quantity. Schumacher's noiseless quantum coding theorem states that a source emitting i.i.d. copies of a mixed state $\rho$ can be compressed, asymptotically, to $S(\rho)$ qubits per copy, and no lower rate achieves high-fidelity reconstruction. This is the direct quantum generalization of Shannon's source coding theorem, and it certifies that von Neumann entropy is the true information content of a quantum source, not merely a convenient formula.

Consider a two-qubit pure state that is deliberately not symmetric between the parties, so both the numbers and the reasoning are non-trivial:

This is already in Schmidt form with Schmidt coefficients $p_1 = 0.7$, $p_2 = 0.3$ (partial entanglement, not maximal). We trace through the algorithm exactly.

Step 1: Full density matrix. In basis $\{|00\rangle, |01\rangle, |10\rangle, |11\rangle\}$,

This matrix has rank 1 (it is $|\psi\rangle\langle\psi|$), eigenvalues $\{1, 0, 0, 0\}$, so $S(\rho_{AB}) = 0$ — correctly reflecting that the global state is pure.

Step 2: Partial trace over $B$. Using the `PartialTrace` routine with $d_A = d_B = 2$:

Step 3: Diagonalize. $\rho_A$ is already diagonal: eigenvalues $\lambda_1 = 0.7$, $\lambda_2 = 0.3$.

Step 4: Entropy.

Step 5: Verify symmetry. Tracing out $A$ instead gives $\rho_B = \begin{pmatrix} 0.7 & 0 \\ 0 & 0.3\end{pmatrix}$ by an identical computation (the state is symmetric under $A \leftrightarrow B$ swap here, so this check is automatic, but in general the Schmidt argument guarantees $S(\rho_A) = S(\rho_B)$ even when $\rho_A \ne \rho_B$ as matrices).

Interpretation table, comparing this state against reference points:

The last row is the crucial contrast: swapping a coherent superposition for a classical probabilistic mixture with the same weights changes $S(\rho_{AB})$ from 0 to 0.881 bits while $S(\rho_A)$ stays at 0.881 bits — because now the global state is itself mixed (no coherence, no entanglement), and $S(\rho_{AB}) = S(\rho_A) + S(\rho_B) - 0 = 0.881$ additively as expected for uncorrelated randomness replicated on both sides via the classical correlation. This distinction — entangled superposition (global pure, local mixed) versus classical correlation (global mixed, marginals mixed) — is exactly what von Neumann entropy is designed to separate, and no purely classical statistic can make this distinction because classical probability theory has no analogue of the density matrix's off-diagonal coherences.

Von Neumann entropy is the quantitative backbone of entanglement verification in real laboratories: experiments certifying entangled photon pairs, trapped ions, or superconducting qubit pairs routinely reconstruct reduced density matrices via tomography and report entanglement entropy as the headline number. In quantum key distribution protocols building on Entanglement as a Physical Resource, security proofs bound an eavesdropper's information using entropic quantities directly descended from von Neumann entropy, most notably in the Devetak–Winter secret-key-rate formula. Quantum error-correcting codes and fault-tolerant architectures use entropy to characterize how noise, modeled through Quantum Channels and Operations, degrades encoded information, since a channel's action on entropy diagnoses whether it is entangling, decohering, or information-preserving. In condensed matter and high-energy theory, entanglement entropy scaling laws (area law versus volume law) classify quantum phases of matter and appear in holographic computations of black hole entropy via the Ryu–Takayanagi formula, making this single quantum-information quantity a genuine bridge between quantum computing, condensed matter physics, and quantum gravity.

Ask the tutor: How does von Neumann entropy change when a state passes through a noisy quantum channel, and why does this connect to the data-processing inequality?