Positive Operators and Square Roots

Positive operators are the linear-algebraic backbone of many quantitative disciplines. In statistics, a covariance matrix of any real random vector is always positive (semidefinite): if $X$ is a random vector with covariance $\Sigma = \mathrm{Cov}(X)$, then for any vector $v$, $\langle \Sigma v, v\rangle = \mathrm{Var}(v^{\mathsf T}X) \geq 0$, exactly matching the defining inequality here; multivariate Gaussian sampling and Mahalanobis distance both rely on the existence of $\Sigma^{1/2}$. In quantum mechanics and quantum information theory, physical observables are modeled by self-adjoint operators, and a density matrix describing the state of a quantum system is required to be a positive operator with trace $1$; positivity encodes that measurement probabilities (given by eigenvalues) are never negative. In numerical linear algebra, the Cholesky decomposition $T = R^*R$ (with $R$ upper triangular rather than the positive square root) is the computational workhorse for solving positive-definite linear systems efficiently, and it exists precisely because of characterization (e) in the theorem above. In pure linear algebra, this chapter is the direct stepping stone to the Singular Value Decomposition, which is obtained by applying the square-root construction to $T^*T$ for an arbitrary (not necessarily self-adjoint) operator $T$, and to the Polar Decomposition, which writes any operator as (isometry)$\times$(positive operator), generalizing the factorization of a complex number into modulus and phase — the positive operator there is exactly $\sqrt{T^*T}$.

Ask the tutor: How does the positive square root of $T^*T$ give rise to the singular values and vectors used in the Singular Value Decomposition?

Think first about the real numbers. A number $x \in \mathbb{R}$ is nonnegative exactly when it can be written as $y^2$ for some real $y$, exactly when it has a nonnegative square root, and — trivially — exactly when $x \geq 0$. Positive operators transplant this entire circle of ideas into linear algebra, replacing "number" with "self-adjoint operator" and "square" with "$T^*T$" or "$S^2$." The bridge that makes this transplant work is the Spectral Theorem for Self-Adjoint Operators. It says every self-adjoint operator $T$ on a finite-dimensional inner product space has an orthonormal basis of eigenvectors $e_1, \dots, e_n$ with real eigenvalues $\lambda_1, \dots, \lambda_n$. In that basis, $T$ acts diagonally:

Positivity of $T$ becomes a statement purely about the eigenvalues: $T \geq 0$ if and only if every $\lambda_i \geq 0$. Once you see $T$ as a diagonal matrix of real numbers in the right basis, all of the equivalences in the main theorem become the corresponding elementary facts about individual nonnegative real numbers, applied coordinatewise. The square root is built the same way: replace each eigenvalue $\lambda_i \geq 0$ by $\sqrt{\lambda_i} \geq 0$, keeping the same eigenvectors. The picture below shows this "coordinatewise" square-rooting: an operator with three nonnegative eigenvalues, and its square root, which stretches each eigenspace by the square root of the original stretch factor.

Example 1: A positive operator on $\mathbb{C}^2$ built as $R^*R$, with its square root computed from scratch. Let $V = \mathbb{C}^2$ with the standard inner product, and let $R \in \mathcal{L}(\mathbb{C}^2)$ have matrix

(an intentionally non-normal, non-self-adjoint matrix, so $T = R^*R$ is not obviously diagonal at a glance). By characterization (e), $T := R^*R$ is automatically positive. Compute:

Check self-adjointness directly: $T$ is a real symmetric matrix, so $T^* = T^{\mathsf T} = T$. Good — consistent with the theorem.

Find the eigenvalues. The characteristic polynomial is

By the quadratic formula,

Both roots are real (consistent with self-adjointness) and both are positive: $3 - 2\sqrt{2} \approx 3 - 2.828 = 0.172 > 0$, and $3+2\sqrt{2} \approx 5.828 > 0$. This confirms (b): all eigenvalues nonnegative, as guaranteed since $T = R^*R$ is positive.

Find eigenvectors. For $\lambda_1 = 3 + 2\sqrt 2$: solve $(T - \lambda_1 I)v = 0$, i.e.

Using the first row: $(1-\lambda_1)v_1 + 2v_2 = 0 \Rightarrow v_2 = \frac{\lambda_1 - 1}{2}v_1 = \frac{2 + 2\sqrt2}{2}v_1 = (1+\sqrt2)v_1$. So an eigenvector is $u_1 = (1,\ 1+\sqrt2)$, with $\|u_1\|^2 = 1 + (1+\sqrt2)^2 = 1 + 1 + 2\sqrt2 + 2 = 4 + 2\sqrt2$. For $\lambda_2 = 3 - 2\sqrt2$: similarly $v_2 = (1-\sqrt2)v_1$, giving eigenvector $u_2 = (1,\ 1-\sqrt2)$, with $\|u_2\|^2 = 1 + (1-\sqrt2)^2 = 4 - 2\sqrt2$. Check orthogonality (as guaranteed by the spectral theorem, since $\lambda_1 \neq \lambda_2$):

Normalize: $e_1 = u_1/\sqrt{4+2\sqrt2}$, $e_2 = u_2 / \sqrt{4-2\sqrt2}$.

Construct $\sqrt T$. By the corollary, $\sqrt T e_i = \sqrt{\lambda_i}\, e_i$. We can write $\sqrt T$ in matrix form using the spectral projection formula

where $P_i$ is the orthogonal projection onto $\mathbb{C}e_i$, i.e. $P_i = e_ie_i^*/\langle e_i,e_i\rangle$ (already normalized so $P_i = e_i e_i^*$). Numerically,

since $3 + 2\sqrt2 = 2 + 2\sqrt2 + 1 = (\sqrt2+1)^2$ exactly — a clean simplification. Similarly

since $3 - 2\sqrt2 = 2 - 2\sqrt2 + 1 = (\sqrt2 -1)^2$ and $\sqrt2 - 1 > 0$. As a sanity check on the whole construction, verify the trace and determinant identities that any square root must satisfy. We know $\mathrm{tr}(\sqrt T) = \sqrt{\lambda_1}+\sqrt{\lambda_2} = (\sqrt2+1)+(\sqrt2-1) = 2\sqrt2 \approx 2.828$, and $\det(\sqrt T) = \sqrt{\lambda_1 \lambda_2} = \sqrt{\det T} = \sqrt{(3+2\sqrt2)(3-2\sqrt2)} = \sqrt{9 - 8} = \sqrt{1} = 1$. These are consistent with $(\sqrt T)^2 = T$ having $\mathrm{tr}(T) = \lambda_1+\lambda_2 = 6$ and $\det(T) = \lambda_1\lambda_2 = 9 - 8 = 1$, both of which match the entries of $T = \begin{pmatrix}1&2\\2&5\end{pmatrix}$: trace $1+5=6$ ✓, determinant $1\cdot5 - 2\cdot2 = 1$ ✓.

Example 2: Distinguishing a positive operator from a self-adjoint operator that is not positive, and showing the square-root construction fails without positivity. Let $T \in \mathcal{L}(\mathbb{R}^3)$ (real inner product space) have matrix, with respect to the standard orthonormal basis,

This $T$ is self-adjoint (real symmetric, in fact already diagonal), with eigenvalues $4, -1, 9$ and standard basis vectors $e_1,e_2,e_3$ as eigenvectors. Check condition (a) directly on $v = e_2 = (0,1,0)$:

So $T$ is not positive — condition (a) fails, and correspondingly condition (b) fails since $-1$ is a negative eigenvalue. Now attempt to build a square root the way the proof of (b) $\Rightarrow$ (c) prescribes: set $S e_i = \sqrt{\lambda_i}\, e_i$. This fails outright for $i=2$ since $\sqrt{-1}$ is not a real number — there is no self-adjoint (real-eigenvalue) operator $S$ with $S^2 e_2 = -e_2$, because any self-adjoint operator has only nonnegative squared-eigenvalues along its own eigenbasis in the following precise sense: if $Sw = \nu w$ with $\nu \in \mathbb R$, then $S^2 w = \nu^2 w$ with $\nu^2 \geq 0$, so $S^2$ can never have a negative eigenvalue. Since $T$ has eigenvalue $-1 < 0$, no self-adjoint square root of $T$ can exist — confirming, from a different angle, the necessity of nonnegative eigenvalues in the theorem. (A non-self-adjoint complex square root does exist if we complexify, e.g. multiplying the $e_2$ direction by $i$, but that lies outside $\mathcal L(\mathbb R^3)$ and outside the scope of positive square roots entirely.) This example illustrates precisely why hypothesis (b) — nonnegativity of eigenvalues — cannot be dropped from the equivalence, a point made rigorous in the counterexample below.