Linear Independence

Linear independence is the backbone of dimension theory: a Basis is defined as a spanning list that is also linearly independent, and the Steinitz Exchange Lemma uses independence to prove that all bases of a finite-dimensional space have the same size — the invariant we call dimension. In numerical linear algebra, checking independence (via the rank of a matrix, Gaussian elimination, or the QR decomposition) underlies the diagnosis of singular systems and the detection of multicollinearity in statistical regression models. In signal processing and coding theory, independent vectors correspond to non-redundant basis functions (e.g., Fourier or wavelet bases), and a failure of independence signals wasted degrees of freedom. In cryptography, independence of vectors over finite fields underlies the analysis of linear codes and secret-sharing schemes, where the rank of certain matrices determines whether a set of shares can reconstruct a secret. In physics, independent eigenvectors of an operator (when they exist) diagonalize the operator, a fact fundamental to quantum mechanics and to the stability analysis of dynamical systems.

Ask the tutor: How does linear independence of a list translate into a rank condition on a matrix, and why does that give a practical algorithm for testing independence?

Picture vectors in $\mathbb{R}^2$ as arrows from the origin. Two vectors are linearly dependent precisely when one is a scalar multiple of the other — they lie on the same line through the origin, so together they only ever sweep out that one line, no matter how you combine them. Two vectors are independent precisely when they point along genuinely different directions; combining them with all possible coefficients lets you reach every point in the plane. The defining equation $a_1v_1 + \cdots + a_nv_n = 0$ is a search for a "conspiracy" among the vectors: a nontrivial way for scaled copies of them to cancel each other out perfectly. If no such conspiracy exists, the vectors are independent. If one does exist, you can always solve that relation for a vector with nonzero coefficient, and express it in terms of the others — this is the algebraic meaning of "redundant."

Note in the left picture that $v_2 = \tfrac{3}{2}v_1$, so $\tfrac{3}{2}v_1 - v_2 = 0$ is a nontrivial relation, and every combination $a_1v_1+a_2v_2$ collapses onto the single line through the origin containing both arrows. In the right picture, $v_1=(2,1)$ and $v_2=(1,-1)$ point in different directions and, as we verify below, no nontrivial relation exists between them; their combinations fill the whole plane.

Example 1: An independence check in $\mathbb{R}^3$ requiring real computation. Let $v_1 = (1, 2, -1)$, $v_2 = (3, 0, 2)$, $v_3 = (-1, 4, -4)$ in $\mathbb{R}^3$. Is $(v_1, v_2, v_3)$ linearly independent? We seek all scalars $a_1, a_2, a_3 \in \mathbb{R}$ with $a_1 v_1 + a_2 v_2 + a_3 v_3 = 0$. Componentwise:

From the second equation: $2a_1 + 4a_3 = 0 \Rightarrow a_1 = -2a_3$. Substitute into the first equation: $(-2a_3) + 3a_2 - a_3 = 0 \Rightarrow 3a_2 = 3a_3 \Rightarrow a_2 = a_3$. Check against the third equation: $-a_1 + 2a_2 - 4a_3 = -(-2a_3) + 2(a_3) - 4a_3 = 2a_3 + 2a_3 - 4a_3 = 0$. ✓ (automatically satisfied — consistent with the first two). So for any choice of $a_3 = t \in \mathbb{R}$, setting $a_1 = -2t$, $a_2 = t$, $a_3 = t$ gives a solution. Taking $t = 1$:

Indeed $-2v_1+v_2+v_3=0$ is a nontrivial relation ($a_1=-2\ne0$). Hence $(v_1,v_2,v_3)$ is linearly dependent. By the theorem's direction (a) $\Rightarrow$ (b), we can solve for any vector with nonzero coefficient in terms of the others; e.g. solving $-2v_1+v_2+v_3=0$ for $v_2$:

confirming $v_2 \in \operatorname{span}(v_1,v_3)$, so $v_2$ is redundant and $\operatorname{span}(v_1,v_2,v_3)=\operatorname{span}(v_1,v_3)$.

Example 2: Confirming independence with a genuinely non-symmetric example, in $\mathbb{R}^2$ and in a function space. Part A ($\mathbb{R}^2$). Take $v_1=(2,1)$, $v_2=(1,-1)$ from the earlier figure. Suppose $a_1(2,1) + a_2(1,-1) = (0,0)$. This gives

The second equation gives $a_1 = a_2$. Substituting into the first: $2a_1 + a_1 = 0 \Rightarrow 3a_1 = 0 \Rightarrow a_1 = 0$, and then $a_2 = a_1 = 0$. The only solution is trivial, so $(v_1,v_2)$ is linearly independent — matching the picture, where the two arrows span the whole plane. Part B (function space). Let $V =$ the vector space of real-valued functions on $\mathbb R$, and let

Suppose $a_1 f_1 + a_2 f_2 + a_3 f_3 = 0$ as functions, i.e. $a_1 + a_2 x + a_3 e^x = 0$ for all $x \in \mathbb{R}$. Differentiate twice (valid since the equation holds identically, so all derivatives of both sides agree): the first derivative gives $a_2 + a_3 e^x = 0$ for all $x$, and the second derivative gives $a_3 e^x = 0$ for all $x$. Since $e^x \ne 0$ for every real $x$, we get $a_3 = 0$. Substituting back into $a_2 + a_3e^x=0$ gives $a_2 = 0$. Substituting both into the original equation gives $a_1 = 0$ for all $x$ (e.g. at $x=0$: $a_1 + 0 + 0 = 0$). Hence $a_1=a_2=a_3=0$ is forced, and $(f_1,f_2,f_3)$ is linearly independent in the (infinite-dimensional) space of functions. This example illustrates that the definition of linear independence makes sense verbatim in any vector space over any field — not just in $\mathbb{F}^n$ — as long as "$=0$" means equality of the zero vector of that space (here, the zero function).