Tensor Products and Multilinear Maps

One-line statement: The tensor product

V \otimes W

is the universal vector space through which all bilinear maps out of

V \times W

factor uniquely as linear maps. Key result in words: Multilinear algebra is linearized — every multilinear map becomes an ordinary linear map on a single tensor space, turning analysis of

k

-fold linear dependence into the study of one big vector space. Where it applies: Differential geometry (tensor fields), representation theory, quantum mechanics (entanglement), homological algebra, algebraic geometry, machine learning (tensor decompositions). Who/when: Roots in Grassmann (1844) and Ricci–Levi-Civita (1900); abstracted by Whitney, Bourbaki, and category theory in the 20th century.

Given vector spaces $V$ and $W$ over a field $k$ , the tensor product $V \otimes_{k} W$ is a vector space equipped with a bilinear map $\otimes : V \times W \to V \otimes_{k} W$ that is universal: any bilinear map $B : V \times W \to U$ factors uniquely through a linear map $\tilde{B} : V \otimes_{k} W \to U$ . This construction matters because it converts multilinearity — an awkward, non-additive notion — into ordinary linearity, allowing the entire machinery of linear algebra to be applied to bilinear, trilinear, and $k$ -linear phenomena. One surprising fact: not every element of $V \otimes W$ is a simple tensor $v \otimes w$ ; generic elements are sums $\sum_{i} v_{i} \otimes w_{i}$ , and the minimum number of terms is the tensor rank, whose exact computation is NP-hard even over $R$ .

The Core Insight

Bilinearity is not preservation of structure under a single operation — a bilinear map fails to be a homomorphism of vector spaces because $B (v_{1} + v_{2}, w) = B (v_{1}, w) + B (v_{2}, w)$ but $B$ is not additive jointly. The tensor product is the precise device that records exactly the relations forced by bilinearity and no more. It is the "freest" target: you build the largest space in which $v \otimes w$ obeys only the bilinear identities, then every other bilinear map is a quotient of this universal one. The slogan: tensor products turn bilinear maps into linear maps.

Formal Statement

Multilinear Map

Let $V_{1}, \dots, V_{k}, U$ be vector spaces over a field $k$ . A map $f : V_{1} \times \dots \times V_{k} \to U$ is multilinear ( $k$ -linear) if it is linear in each argument separately: for each $i$ , with all other arguments fixed,

f (v_{1}, \dots, α v_{i} + β v_{i}^{'}, \dots, v_{k}) = α f (\dots, v_{i}, \dots) + β f (\dots, v_{i}^{'}, \dots) .

⟳this is just

k

copies of the single-variable linearity condition, one per slot, with all other slots frozen.

For $k = 2$ this is bilinear. The set of all $k$ -linear maps $V_{1} \times \dots \times V_{k} \to U$ is a vector space, denoted $Mult (V_{1}, \dots, V_{k}; U)$ or $L^{k} (V_{1}, \dots, V_{k}; U)$ .

Tensor Product via Universal Property

A tensor product of $V$ and $W$ is a pair $(T, \otimes)$ consisting of a vector space $T$ and a bilinear map $\otimes : V \times W \to T$ , written $(v, w) \mapsto v \otimes w$ , satisfying the universal property: for every vector space $U$ and every bilinear map $B : V \times W \to U$ , there exists a unique linear map $\tilde{B} : T \to U$ with

B (v, w) = \tilde{B} (v \otimes w) for all v \in V, w \in W .

⟳this is the factorization

B = \tilde{B} \circ \otimes

; uniqueness comes because simple tensors

v \otimes w

span

T

, so

\tilde{B}

is determined on a spanning set.

We write $T = V \otimes_{k} W$ . The defining bilinear identities are

(v + v^{'}) \otimes w = v \otimes w + v^{'} \otimes w, v \otimes (w + w^{'}) = v \otimes w + v \otimes w^{'}, (α v) \otimes w = α (v \otimes w) = v \otimes (α w) .

⟳these follow from requiring

\otimes

itself to be bilinear; they are the only relations imposed in the free construction below.

Explicit Construction

Let $F (V \times W)$ be the free vector space on the set $V \times W$ (basis = all pairs $(v, w)$ ). Let $R \subseteq F (V \times W)$ be the subspace spanned by all elements

(v + v^{'}, w) - (v, w) - (v^{'}, w), (v, w + w^{'}) - (v, w) - (v, w^{'}), (α v, w) - α (v, w), (v, α w) - α (v, w) .

⟳each generator of

R

is "the failure of bilinearity"; quotienting kills exactly those failures.

Then define

V \otimes_{k} W := F (V \times W) / R, v \otimes w := \overline{(v, w)} (coset of (v, w)) .

⟳the quotient map sends the formal pair to its class; the relations

R

are precisely those of equation [3], so

\otimes

is bilinear and universal by the universal property of free objects and quotients.

Existence, Uniqueness, and Dimension

The tensor product exists and is unique up to unique isomorphism: if $(T, \otimes)$ and $(T^{'}, \otimes^{'})$ both satisfy the universal property, there is a unique linear isomorphism $ϕ : T \to T^{'}$ with $ϕ (v \otimes w) = v \otimes^{'} w$ . Moreover, if ${e_{i}}_{i \in I}$ is a basis of $V$ and ${f_{j}}_{j \in J}$ is a basis of $W$ , then ${e_{i} \otimes f_{j}}_{(i, j) \in I \times J}$ is a basis of $V \otimes_{k} W$ . Hence in finite dimensions

dim (V \otimes_{k} W) = (dim V) (dim W) .

⟳the basis claim follows from the construction in [5] plus the universal property; dimension is the cardinality of the product index set

I \times J

There is a canonical isomorphism

Mult (V, W; U) ≅ L (V \otimes_{k} W, U),

⟳send a bilinear

B

to its unique factorization

\tilde{B}

; this map is linear and invertible by the universal property — it is the categorical meaning of "tensor product is left adjoint to Hom".

Why It Holds / Derivation

Uniqueness up to unique isomorphism

Step 1. Suppose

(T, \otimes)

and

(T^{'}, \otimes^{'})

both satisfy the universal property. Since

\otimes^{'} : V \times W \to T^{'}

is bilinear, by universality of

(T, \otimes)

there is a unique linear

ϕ : T \to T^{'}

with

ϕ \circ \otimes = \otimes^{'}

, i.e.

ϕ (v \otimes w) = v \otimes^{'} w

. Step 2. Symmetrically, since

\otimes : V \times W \to T

is bilinear, universality of

(T^{'}, \otimes^{'})

yields a unique linear

ψ : T^{'} \to T

with

ψ (v \otimes^{'} w) = v \otimes w

. Step 3. Consider

ψ \circ ϕ : T \to T

. It satisfies

(ψ \circ ϕ) (v \otimes w) = v \otimes w

, so

ψ \circ ϕ

and

id_{T}

are two linear maps factoring the bilinear map

\otimes

through

T

. By the uniqueness clause of the universal property,

ψ \circ ϕ = id_{T}

. Symmetrically

ϕ \circ ψ = id_{T^{'}}

. Step 4. Hence

ϕ

is the unique isomorphism intertwining the universal maps.

■

The $e_i\otimes f_j$ form a basis (finite-dim case)

Step 1 (Spanning). Any

v = \sum_{i} a_{i} e_{i}

w = \sum_{j} b_{j} f_{j}

give, by bilinearity [3],

v \otimes w = \sum_{i, j} a_{i} b_{j} (e_{i} \otimes f_{j})

. Since simple tensors span

V \otimes W

(true in the construction [5]), the

e_{i} \otimes f_{j}

span. Step 2 (Independence). For each pair

(p, q)

define the bilinear map

B_{pq} : V \times W \to k

B_{pq} (v, w) = a_{p} b_{q}

(coordinate extraction). By universality there is a linear

\tilde{B}_{pq} : V \otimes W \to k

with

\tilde{B}_{pq} (e_{i} \otimes f_{j}) = δ_{i p} δ_{j q}

. Step 3. Suppose

\sum_{i, j} c_{ij} e_{i} \otimes f_{j} = 0

. Apply

\tilde{B}_{pq}

\sum_{i, j} c_{ij} δ_{i p} δ_{j q} = c_{pq} = 0

for all

(p, q)

. Hence linear independence.

■

:::proof[The Hom-tensor adjunction, equation [7]] Step 1. Define $Φ : Mult (V, W; U) \to L (V \otimes W, U)$ by $Φ (B) = \tilde{B}$ , the factorization guaranteed by [2]. Linearity of $Φ$ : $(α B_{1} + β B_{2})$ agrees with $α \tilde{B}_{1} + β \tilde{B}_{2}$ on simple tensors, hence everywhere.

Step 2. Define

Ψ : L (V \otimes W, U) \to Mult (V, W; U)

Ψ (L) (v, w) = L (v \otimes w)

. This is bilinear in

(v, w)

since

\otimes

is. Step 3.

Ψ (Φ (B)) (v, w) = \tilde{B} (v \otimes w) = B (v, w)

, so

ΨΦ = id

. And

Φ (Ψ (L))

is the unique factorization of

(v, w) \mapsto L (v \otimes w)

, which is

L

itself. So

ΦΨ = id

■

Hermann Grassmann | 1809–1877

Grassmann's Ausdehnungslehre (Theory of Extension, 1844, revised 1862) introduced the exterior (Grassmann) product and the conceptual seed of multilinear algebra decades before the tensor formalism crystallized. Working in near-total obscurity as a Gymnasium teacher in Stettin, he developed an algebra of "extensive quantities" — what we now recognize as elements of exterior and tensor algebras — including the alternating product $v \land w$ . His work was so far ahead of its notation that few contemporaries understood it; only after Ricci-Curbastro and Levi-Civita's "absolute differential calculus" (tensor analysis, c. 1900) and its dramatic use by Einstein in general relativity did the full architecture of multilinear algebra gain recognition. The modern universal-property definition is due to the categorical reformulation by Bourbaki and others in the mid-20th century.

Building Intuition

Think of $V \otimes W$ as the place where you can multiply vectors from different spaces without choosing coordinates. You cannot "multiply" a velocity vector by a force vector inside $R^{3}$ — the product $v \otimes w$ lives in a larger $9$ -dimensional space, and the entries $v_{i} w_{j}$ are exactly the components.

A cleaner mental model: $V \otimes W ≅$ matrices when $V = k^{m}$ , $W = k^{n}$ . A simple tensor $v \otimes w$ is the rank-one matrix $v w^{⊤}$ (outer product). A general element is any matrix, written as a sum of rank-one pieces. The minimal number of summands is the matrix rank — and this is the prototype for the deep notion of tensor rank in higher orders. Bilinearity of $\otimes$ corresponds to the fact that $(v_{1} + v_{2}) w^{⊤} = v_{1} w^{⊤} + v_{2} w^{⊤}$ .

The universal property is the abstract way of saying: to define a linear map out of $V \otimes W$ , it suffices and is necessary to specify a bilinear map on $V \times W$ . You never need to check what happens on general sums — only on the building blocks $v \otimes w$ , and only that the bilinear relations are respected.

Universal property of the tensor product: every bilinear $B$ factors uniquely through $\otimes$ via a linear $\tilde B$.

Worked Examples

Outer product as a simple tensor in $\mathbb{R}^2\otimes\mathbb{R}^3$

Given:

v = (1, 2)^{⊤} \in R^{2}

with basis

{e_{1}, e_{2}}

, and

w = (3, 0, - 1)^{⊤} \in R^{3}

with basis

{f_{1}, f_{2}, f_{3}}

. Compute

v \otimes w

in the basis

{e_{i} \otimes f_{j}}

and as a matrix. Step 1. Expand by bilinearity [3]:

v \otimes w = (e_{1} + 2 e_{2}) \otimes (3 f_{1} + 0 f_{2} - f_{3}) .

Step 2. Distribute: coefficients are

c_{ij} = v_{i} w_{j}

$c_{11} = 1 \cdot 3 = 3$ , $c_{12} = 1 \cdot 0 = 0$ , $c_{13} = 1 \cdot (- 1) = - 1$
$c_{21} = 2 \cdot 3 = 6$ , $c_{22} = 0$ , $c_{23} = 2 \cdot (- 1) = - 2$

Step 3. As the outer product matrix

v w^{⊤}

(rows indexed by

i

, columns by

j

v \otimes w = (3600 - 1 - 2) = 3 e_{1} \otimes f_{1} - e_{1} \otimes f_{3} + 6 e_{2} \otimes f_{1} - 2 e_{2} \otimes f_{3}

This matrix has rank 1, confirming $v \otimes w$ is a simple tensor (dimensionless coefficients).

A tensor that is NOT simple (tensor rank 2)

Given:

t = e_{1} \otimes f_{1} + e_{2} \otimes f_{2} \in R^{2} \otimes R^{2}

. Decide whether

t = v \otimes w

for some

v, w

. Step 1. As a matrix,

t = (1001)

(the identity). Step 2. A simple tensor corresponds to a rank-1 matrix

v w^{⊤}

, whose determinant is

0

(any

2 \times 2

outer product has linearly dependent columns). Step 3. But

det (1001) = 1 \neq = 0

, so

t

has matrix rank

2

. Step 4. Therefore

t

cannot be written as a single

v \otimes w

t = e_{1} \otimes f_{1} + e_{2} \otimes f_{2} has tensor rank 2; it is NOT a simple tensor.

This $t$ is exactly the maximally entangled Bell state $\frac{1}{2} (∣00 ⟩ + ∣11 ⟩)$ up to normalization — the prototype of quantum entanglement.

Common Misconceptions

Pitfall 1: "Every element of $V \otimes W$ is $v \otimes w$ ." False. Simple tensors only span

V \otimes W

; the generic element is a sum, and (Example 2) need not collapse to a single term. The minimal number of terms is the tensor rank. Pitfall 2: " $dim (V \otimes W) = dim V + dim W$ ." That is the direct sum

V \oplus W

. For tensor product the dimension multiplies:

dim (V \otimes W) = dim V \cdot dim W

(equation [6]). Pitfall 3: " $v \otimes w = w \otimes v$ ." Only after choosing the swap isomorphism

V \otimes W ≅ W \otimes V

; these are different spaces and the elements are not literally equal. The braiding

v \otimes w \mapsto w \otimes v

is canonical but nontrivial (it is the source of symmetric vs. exterior algebra). Pitfall 4: Forgetting the field.

V \otimes_{k} W

depends on

k

. For example

C \otimes_{R} C ≅ C \oplus C

(real dimension 4), whereas

C \otimes_{C} C ≅ C

(dimension 1 over

C

Tensor product is not exact on the left over $\mathbb{Z}$

Over a non-field ring the tensor product can fail to be exact. Consider $Z$ -modules. The injection $Z \times 2 Z$ is injective, but tensoring with $Z /2 Z$ gives

Z /2 \times 2 Z /2,

which is the zero map, hence not injective. So

- \otimes Z /2

is only right exact, not exact; its failure is measured by

Tor_{1}

. Over a field this never happens (all modules are free/flat), which is exactly why the clean dimension count [6] holds for vector spaces but not for general modules.

Connections

The tensor product is the hub of multilinear algebra and threads into many areas:

Exterior Algebra: $Λ^{k} V$ is the quotient of $V^{\otimes k}$ by the subspace generated by $v_{1} \otimes \dots \otimes v_{k}$ with a repeated factor; it linearizes alternating multilinear maps (determinants, differential forms).
Symmetric Algebra: $Sym^{k} V = V^{\otimes k} / (transposition relations)$ linearizes symmetric multilinear maps (polynomials).
Dual Space and Bilinear Forms: $V^{*} \otimes W ≅ Hom (V, W)$ in finite dimensions, and $V^{*} \otimes V^{*} ≅$ bilinear forms on $V$ .
Category Theory and Adjoint Functors: equation [7] expresses the adjunction $- \otimes W ⊣ Hom (W, -)$ , making $(Vect_{k}, \otimes, k)$ a symmetric monoidal category.
Tensor Fields on Manifolds: pointwise tensor products of tangent and cotangent spaces give the global tensor bundle in differential geometry.

Real-World Applications

General Relativity — The metric $g_{μν}$ , Riemann curvature $R^{ρ}_{σ μν}$ , and stress-energy $T_{μν}$ are sections of tensor bundles built from tensor products of $T_{p} M$ and $T_{p}^{*} M$ ; Einstein's field equations are a tensor identity.

Quantum Mechanics & Quantum Computing — The state space of a composite system is the tensor product $H_{A} \otimes H_{B}$ . Entangled states are precisely the non-simple tensors (Example 2); $n$ qubits live in $(C^{2})^{\otimes n}$ of dimension $2^{n}$ .

Machine Learning — Tensor Decompositions — CP (CANDECOMP/PARAFAC) and Tucker decompositions express data tensors as sums of simple tensors; tensor rank governs model complexity and is central to neural-network compression and signal separation.

Continuum Mechanics — The Cauchy stress and strain tensors are rank-2 tensors over $R^{3}$ ; the elasticity (stiffness) tensor $C_{ij k l}$ is a rank-4 tensor linking them via Hooke's law $σ_{ij} = C_{ij k l} ε_{k l}$ .

Representation Theory & Particle Physics — Tensor products of group representations decompose into irreducibles (Clebsch–Gordan); coupling of angular momenta and quark composition of hadrons (e.g. $3 \otimes \overset{ˉ}{3} = 1 \oplus 8$ for mesons) are tensor-product statements.

Algebraic Geometry / Commutative Algebra — Fiber products of schemes correspond to tensor products of rings, $Spec (A \otimes_{R} B)$ ; base change and extension of scalars are tensor operations.

Quantum Chemistry & Tensor Networks — Matrix Product States (MPS) and DMRG represent many-body wavefunctions in huge tensor-product Hilbert spaces by chains of small tensors, exploiting low entanglement.

Historical Development

1844 — Hermann Grassmann publishes Die lineale Ausdehnungslehre, introducing exterior products and the algebra of extensive quantities; largely ignored.
1862 — Grassmann's revised Ausdehnungslehre, with a cleaner exposition of multilinear products.
1884–1885 — J. Willard Gibbs develops dyadics (essentially rank-2 tensors as $\sum v_{i} \otimes w_{i}$ ) in his lectures on vector analysis.
1890 — Woldemar Voigt coins the word tensor in the context of elasticity (the stress tensor).
1900 — Gregorio Ricci-Curbastro and Tullio Levi-Civita publish Méthodes de calcul différentiel absolu, the systematic tensor calculus.
1915–1916 — Einstein's general relativity makes tensor analysis indispensable to physics, popularizing index notation.
1938 — Hassler Whitney gives a rigorous abstract construction of the tensor product of abelian groups/modules ("Tensor products of Abelian groups").
1940s–1950s — Bourbaki (Algèbre) formalizes the universal-property definition for modules over a ring, the modern standard.
1958–1965 — Categorical viewpoint (Eilenberg, Mac Lane, Kelly): tensor products characterized via adjunction and monoidal categories.

Subtleties & Limits

Tensor rank and complexity. While matrix rank (order-2 tensors) is computable by Gaussian elimination, for order

\geq 3

the situation is wild:

Computing tensor rank is NP-hard (Håstad, 1990) over $Q$ and $F_{q}$ .
Rank depends on the field: a real tensor can have higher rank over $R$ than over $C$ .
Border rank vs. rank: a tensor can be a limit of rank- $r$ tensors without itself having rank $r$ (the set of bounded-rank tensors is not closed), which underlies fast matrix multiplication.
Matrix multiplication exponent $ω$ : the bilinear map $M_{n} \times M_{n} \to M_{n}$ is a tensor whose (border) rank growth determines $ω$ ; whether $ω = 2$ is a major open problem. Strassen's 1969 algorithm exploits that the $2 \times 2$ multiplication tensor has rank $\leq 7 < 8$ .
Maximal and generic ranks of tensor spaces are unknown in general dimensions; even the maximal rank of $R^{3} \otimes R^{3} \otimes R^{3}$ involves subtle algebraic geometry (secant varieties of Segre embeddings).

Tensor Products and Multilinear Maps

The Core Insight

Formal Statement

Multilinear Map

Tensor Product via Universal Property

Explicit Construction

Existence, Uniqueness, and Dimension

Why It Holds / Derivation

Uniqueness up to unique isomorphism

The $e_i\otimes f_j$ form a basis (finite-dim case)

Hermann Grassmann | 1809–1877

Building Intuition

Universal property of the tensor product: every bilinear $B$ factors uniquely through $\otimes$ via a linear $\tilde B$.

Worked Examples

Outer product as a simple tensor in $\mathbb{R}^2\otimes\mathbb{R}^3$

A tensor that is NOT simple (tensor rank 2)

Common Misconceptions

Tensor product is not exact on the left over $\mathbb{Z}$

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also

Before this

Where this leads