Linear Independence and Basis

Linear independence means no vector in a set can be written as a combination of the others; a basis is a linearly independent set that spans the whole space. Together they pin down a minimal, complete coordinate system: every vector has exactly one expression in terms of basis vectors. This underlies all of linear algebra — dimension, coordinates, rank, and matrix representations. Formalized in the late 19th–early 20th century (Grassmann, Peano, Steinitz).

A set of vectors ${v_{1}, \dots, v_{n}}$ in a vector space $V$ is linearly independent if the only scalars $c_{1}, \dots, c_{n}$ making $c_{1} v_{1} + \dots + c_{n} v_{n} = 0$ are all zero; otherwise it is linearly dependent. A basis is a set that is both linearly independent and spanning, meaning it is simultaneously a maximal independent set and a minimal spanning set. The remarkable structural fact is that every basis of a given space has the same number of elements — this number is the dimension, an invariant of the space itself. From these two intertwined ideas flow coordinates, the rank–nullity theorem, and the entire matrix calculus of linear maps. The surprising fact: even infinite-dimensional spaces have bases (Hamel bases), but only by invoking the Axiom of Choice, and such bases are almost never explicitly writable.

The Core Insight

Independence and spanning are opposing forces. Adding vectors helps you span more but threatens independence; removing vectors preserves independence but threatens spanning. A basis is the exact equilibrium — the unique sweet spot where you have just enough vectors to reach everything, with no redundancy. Because of this, basis coordinates are unique, turning abstract vectors into concrete tuples of numbers.

Formal Statement

Linear Combination and Span

Let $V$ be a vector space over a field $F$ (typically $R$ or $C$ ). Given vectors $v_{1}, \dots, v_{n} \in V$ , a linear combination is any vector of the form

c_{1} v_{1} + c_{2} v_{2} + \dots + c_{n} v_{n}, c_{i} \in F .

⟳this is the closure of

V

under the two vector-space operations (addition and scalar multiplication) applied finitely to the given vectors.

The span is the set of all such combinations:

span {v_{1}, \dots, v_{n}} = {i = 1 \sum n c_{i} v_{i} : c_{i} \in F} .

⟳collect every reachable vector; closure under

+

and scalar

\cdot

makes the span a subspace of

V

Linear Independence

The set ${v_{1}, \dots, v_{n}}$ is linearly independent if

c_{1} v_{1} + c_{2} v_{2} + \dots + c_{n} v_{n} = 0 ⟹ c_{1} = c_{2} = \dots = c_{n} = 0.

⟳the zero vector must have only the trivial representation; any nontrivial solution exhibits dependence.

If a nontrivial solution exists, the set is linearly dependent, and at least one vector is a combination of the others:

v_{k} = - \frac{1}{c _{k}} i \neq = k \sum c_{i} v_{i} (for any c_{k} \neq = 0) .

⟳isolate a term whose coefficient is nonzero in the dependence relation

\sum c_{i} v_{i} = 0

and divide.

Basis and Dimension

A set $B = {b_{1}, \dots, b_{n}} \subseteq V$ is a basis of $V$ if it is linearly independent and $span (B) = V$ . Every $v \in V$ then has a unique coordinate representation:

v = i = 1 \sum n α_{i} b_{i}, [v]_{B} = (α_{1}, \dots, α_{n}) \in F^{n} .

⟳spanning gives existence of the

α_{i}

; independence gives uniqueness (subtract two representations and apply [3]).

The dimension $dim V = n$ is the common cardinality of every basis of $V$ .

Dimension is Well-Defined (Steinitz Exchange)

If $V$ has a finite spanning set, then every basis of $V$ has the same number of elements. More sharply, if ${u_{1}, \dots, u_{m}}$ is linearly independent and ${w_{1}, \dots, w_{n}}$ spans $V$ , then $m \leq n$ .

Why It Holds / Derivation

Steinitz Exchange Lemma

We show any independent set is no larger than any spanning set, by swapping vectors one at a time.

Step 1. Let

U = {u_{1}, \dots, u_{m}}

be independent and

W = {w_{1}, \dots, w_{n}}

span

V

. Since

W

spans, write

u_{1} = j = 1 \sum n a_{j} w_{j} .

⟳

u_{1} \in V = span (W)

, so coefficients

a_{j}

exist by definition of span.

Step 2. Not all

a_{j}

are zero (else

u_{1} = 0

, contradicting independence). Reindex so

a_{1} \neq = 0

; then solve [6] for

w_{1}

w_{1} = \frac{1}{a _{1}} (u_{1} - j = 2 \sum n a_{j} w_{j}) .

⟳divide [6] by

a_{1} \neq = 0

and isolate

w_{1}

; this shows

w_{1} \in span (u_{1}, w_{2}, \dots, w_{n})

Step 3. Replace

w_{1}

u_{1}

: the set

{u_{1}, w_{2}, \dots, w_{n}}

still spans

V

, because anything expressible using

w_{1}

can be re-expressed via [7]. Step 4. Repeat: express

u_{2}

in the new spanning set; its coefficient on some remaining

w_{j}

must be nonzero (otherwise

u_{2}

would depend on

u_{1}

alone, contradicting independence of

U

). Swap that

w_{j}

for

u_{2}

. Iterating, each

u_{i}

replaces a distinct

w_{j}

. Step 5. If

m > n

we would run out of

w

's to replace before exhausting the

u

's, forcing some

u_{i}

to lie in

span (u_{1}, \dots, u_{i - 1})

— contradicting independence. Hence

m \leq n

■

Corollary. Two bases

B_{1}, B_{2}

are each independent and spanning, so

∣ B_{1} ∣ \leq ∣ B_{2} ∣

and

∣ B_{2} ∣ \leq ∣ B_{1} ∣

, giving

∣ B_{1} ∣ = ∣ B_{2} ∣

Giuseppe Peano | 1858–1932

Peano gave, in his 1888 Calcolo geometrico secondo l'Ausdehnungslehre di H. Grassmann, the first fully axiomatic definition of a vector space over $R$ — including linear independence and dimension in essentially modern form, decades ahead of his contemporaries. He recast Hermann Grassmann's dense Ausdehnungslehre (1844) into clean axioms: closure, associativity, distributivity, and the existence of a basis. Peano's clarity was largely ignored for forty years until the abstract approach was revived by Banach, Weyl, and the development of functional analysis in the 1920s–30s.

Building Intuition

Think of a basis as a complete, non-redundant set of directions in which you are allowed to walk.

Independence = no wasted directions. If one arrow points where you can already get by combining others, it's redundant — remove it.
Spanning = you can reach everywhere. If some point in space is unreachable, you're missing a direction — add one.
Basis = exactly the right arrows. Not one too many, not one too few.

A useful mental test for dependence in $R^{n}$ : vectors are dependent iff they all "fit inside" a lower-dimensional flat through the origin. Two vectors are dependent iff collinear; three are dependent iff coplanar. The dimension counts how many independent directions you genuinely have — and coordinates are just the recipe (how much of each basis direction) to build any given vector.

Two independent vectors span the plane; a third must be dependent

Worked Examples

Testing Independence in $\mathbb{R}^3$

Given:

v_{1} = (1, 2, 1)

v_{2} = (2, 1, 0)

v_{3} = (1, - 1, - 1)

. Are they independent? Do they form a basis of

R^{3}

? Step 1. Set up

c_{1} v_{1} + c_{2} v_{2} + c_{3} v_{3} = 0

, i.e. the matrix equation

A c = 0

with columns

v_{i}

A = 121210 1 - 1 - 1 .

⟳stack each vector as a column; the system

A c = 0

encodes the dependence relation [3].

Step 2. Compute the determinant (independence

\Leftrightarrow det A \neq = 0

for

n

vectors in

R^{n}

det A = 1 (1 \cdot (- 1) - (- 1) \cdot 0) - 2 (2 \cdot (- 1) - (- 1) \cdot 1) + 1 (2 \cdot 0 - 1 \cdot 1) .

⟳cofactor expansion along the first row of [8].

Step 3. Evaluate:

det A = 1 (- 1) - 2 (- 2 - (- 1)) + 1 (- 1) = - 1 - 2 (- 1) - 1 = - 1 + 2 - 1 = 0.

Conclusion:

det A = 0

, so the vectors are linearly dependent. Indeed one checks

v_{3} = v_{2} - v_{1}

... verify:

v_{2} - v_{1} = (1, - 1, - 1) = v_{3}

. ✓

Dependent; NOT a basis. v_{3} = v_{2} - v_{1} .

Coordinates Relative to a Basis

Given: Basis

B = {b_{1}, b_{2}}

R^{2}

with

b_{1} = (1, 1)

b_{2} = (1, - 1)

. Find

[v]_{B}

for

v = (4, 2)

. Step 1. Solve

v = α_{1} b_{1} + α_{2} b_{2}

{α_{1} + α_{2} = 4 α_{1} - α_{2} = 2.

⟳match components of

α_{1} (1, 1) + α_{2} (1, - 1) = (4, 2)

Step 2. Add the equations:

2 α_{1} = 6 \Rightarrow α_{1} = 3

. Subtract:

2 α_{2} = 2 \Rightarrow α_{2} = 1

. Step 3. Verify:

3 (1, 1) + 1 (1, - 1) = (3 + 1, 3 - 1) = (4, 2)

. ✓

[v]_{B} = (3, 1) .

Note these coordinates differ from the standard-basis coordinates $(4, 2)$ — the same vector has different coordinate tuples in different bases.

Common Misconceptions

"More vectors = more spanning power, always good." No. Beyond

dim V

vectors, you cannot increase the span (it's already

V

), and you must lose independence. In

R^{n}

, any set of

n + 1

vectors is automatically dependent — regardless of how cleverly chosen. "A basis is unique." False — there are infinitely many bases of any nontrivial space. What is unique is the number of basis vectors (the dimension) and the coordinates of a vector with respect to a fixed basis. "Independence depends on the order or labeling." Independence is a property of the set, invariant under reordering. Coordinates, however, depend on an ordered basis. "Zero can be in a basis." Never. Any set containing the zero vector is dependent, since

1 \cdot 0 + 0 \cdot (rest) = 0

is a nontrivial relation.

Pairwise independence ≠ joint independence

In $R^{2}$ take $v_{1} = (1, 0)$ , $v_{2} = (0, 1)$ , $v_{3} = (1, 1)$ . Any two of these are linearly independent (no two are collinear). Yet the full set is dependent:

v_{1} + v_{2} - v_{3} = (1, 0) + (0, 1) - (1, 1) = (0, 0) .

⟳a single nontrivial relation among all three suffices for dependence, even when every pair looks fine.

Lesson: Independence is a global property of the whole set, not reducible to checking pairs.

Connections

Linear independence and basis sit at the hub of linear algebra:

Rank-Nullity Theorem — the rank of a matrix is the dimension of its column space (a basis count of the image); nullity is the basis count of the kernel.
Span and Subspaces — bases are minimal spanning sets; every subspace has its own dimension $\leq dim V$ .
Linear Transformations — a linear map is completely determined by its action on a basis, which is exactly why matrices represent maps.
Gram-Schmidt Process — converts any basis into an orthonormal one, marrying independence with inner-product geometry.
Change of Basis — invertible matrices relate coordinates in one basis to another; the columns are the new basis vectors expressed in the old.
Eigenvectors and Eigenvalues — a basis of eigenvectors diagonalizes an operator, the cleanest possible representation.

Real-World Applications

Computer graphics & robotics — object orientation uses a basis of three orthonormal axes; change-of-basis matrices rotate and transform coordinate frames between camera, world, and joint spaces.
Principal Component Analysis (PCA) — finds an orthonormal eigenbasis of the data covariance matrix; dropping low-variance directions compresses data while keeping an independent reduced basis.
Quantum mechanics — states live in a Hilbert space; measurement bases (position, momentum, energy eigenstates) are orthonormal bases, and superposition is literally a linear combination.
Signal processing (Fourier & wavelets) — sines/cosines form a basis for periodic signals; wavelet bases give localized, independent building blocks for compression (JPEG-2000).
Error-correcting codes — a linear code is the row span of a generator matrix; its independent rows form a basis whose dimension equals the number of information bits.
Structural engineering / finite elements — displacement fields are expanded in a basis of shape functions; independence ensures the stiffness matrix is invertible.
Machine learning embeddings — word/feature vectors are coordinates in a learned basis; linear independence of features prevents redundant, collinear inputs that destabilize regression.

Historical Development

1844 — Hermann Grassmann publishes Die lineale Ausdehnungslehre, introducing $n$ -dimensional spaces, linear independence, and dimension — but in notoriously obscure prose.
1862 — Grassmann releases a revised, clearer edition; still largely unread by mainstream mathematicians.
1888 — Giuseppe Peano gives the first axiomatic vector-space definition over $R$ , with explicit treatment of bases and dimension.
1888 — Peano also states what is essentially the dimension invariance result for finite systems.
1898 — Ernst Steinitz (and later in his 1910 field-theory work) formalizes the exchange lemma that guarantees all bases have equal cardinality.
1918 — Hermann Weyl, in Raum, Zeit, Materie, popularizes the abstract vector-space viewpoint within physics and relativity.
1922–1932 — Stefan Banach and others build functional analysis on infinite-dimensional vector spaces, motivating Hamel and Schauder bases.
1934 — Hamel basis existence for arbitrary vector spaces is established as a consequence of the Axiom of Choice (Zorn's Lemma).
1940s–50s — Bourbaki canonizes the modern definitions, making "linearly independent" and "basis" standard undergraduate vocabulary.

Subtleties & Limits

Infinite-dimensional bases are slippery. In an infinite-dimensional space, a Hamel basis (allowing only finite linear combinations) always exists — but only via the Axiom of Choice, and it is essentially never explicitly constructible. A Hamel basis of

R

over

Q

, for instance, is non-measurable and cannot be exhibited.

Analysts therefore prefer a Schauder basis, which permits infinite convergent sums (e.g. the Fourier basis of $L^{2}$ ). Crucially, these are different notions: a Hamel basis of an infinite-dimensional Banach space is always uncountable, while a Schauder basis can be countable. Whether every separable Banach space has a Schauder basis was a famous open question — Per Enflo answered it negatively in 1973, constructing a separable Banach space with no Schauder basis, a result that won him a live goose from Stanisław Mazur who had posed the problem in 1936. The interplay between algebraic (Hamel) and topological (Schauder) bases remains a rich subtlety distinguishing pure linear algebra from functional analysis.