Matrix Multiplication and Dot Products

Matrix multiplication combines two matrices by taking dot products of rows with columns. For matrices

A

(size

m \times n

) and

B

(size

n \times p

), the product

A B

is an

m \times p

matrix whose

(i, j)

entry is the dot product of row

i

A

with column

j

B

. Where it applies: linear systems, geometric transformations, computer graphics, statistics, machine learning. Who/when: formalized by Arthur Cayley (1858), building on work by Jacques Binet and Gotthold Eisenstein.

The dot product takes two vectors of equal length and returns a single number: multiply corresponding entries and add. Matrix multiplication generalizes this by arranging many dot products into a grid. Given $A$ of size $m \times n$ and $B$ of size $n \times p$ , their product $A B$ exists only when the inner dimensions match ( $n = n$ ), and the result has size $m \times p$ . This operation matters because it encodes the composition of linear transformations — applying one transformation after another corresponds to multiplying their matrices. The surprising fact: matrix multiplication is not commutative; $A B$ and $B A$ are usually different, and one may not even exist while the other does.

The Core Insight

A matrix is a machine that transforms vectors. Multiplying matrices means chaining machines together. Each entry of the product is a dot product — a measure of how much one row "agrees" with one column. The dot product is the atom; matrix multiplication is the molecule built from many such atoms arranged in rows and columns.

Formal Statement

Dot Product

Given two vectors of the same length $n$ ,

a = (a_{1}, a_{2}, \dots, a_{n}), b = (b_{1}, b_{2}, \dots, b_{n}),

⟳A vector is just an ordered list of numbers; both must have the same number of components for the dot product to be defined.

their dot product (or scalar product) is the single number

a \cdot b = k = 1 \sum n a_{k} b_{k} = a_{1} b_{1} + a_{2} b_{2} + \dots + a_{n} b_{n} .

⟳Multiply each pair of matching entries, then add all the products together — the result is a scalar, not a vector.

A second, geometric formula relates the dot product to lengths and the angle $θ$ between the vectors:

a \cdot b = ∥ a ∥ ∥ b ∥ cos θ,

⟳Apply the Law of Cosines to the triangle formed by

a

b

, and

a - b

, then expand using equation [2].

where $∥ a ∥ = a \cdot a$ is the length (magnitude) of $a$ .

Matrix Multiplication

Let $A$ be an $m \times n$ matrix with entries $a_{ik}$ and $B$ an $n \times p$ matrix with entries $b_{k j}$ . The product $C = A B$ is the $m \times p$ matrix whose entries are

c_{ij} = k = 1 \sum n a_{ik} b_{k j}, 1 \leq i \leq m, 1 \leq j \leq p .

⟳The entry in row

i

, column

j

of the product is the dot product of row

i

A

with column

j

B

— sum over the shared index

k

Symbols:

$m$ = number of rows of $A$ (and of $C$ ).
$n$ = number of columns of $A$ = number of rows of $B$ (the inner dimension, which must match).
$p$ = number of columns of $B$ (and of $C$ ).
$a_{ik}$ = entry of $A$ in row $i$ , column $k$ .
$b_{k j}$ = entry of $B$ in row $k$ , column $j$ .
$c_{ij}$ = entry of the product in row $i$ , column $j$ .

Compatibility and Size Rule

The product $A B$ is defined if and only if the number of columns of $A$ equals the number of rows of $B$ . In that case:

(m \times n) \cdot (n \times p) ⟶ (m \times p) .

⟳The inner dimensions (

n

and

n

) must cancel; the outer dimensions (

m

and

p

) survive as the result's size.

Computing one entry of a matrix product as a row–column dot product

Why It Holds / Derivation

Why Matrix Multiplication Composes Transformations

Matrix multiplication is defined the way it is so that applying transformation $B$ , then transformation $A$ , equals applying the single matrix $A B$ .

Step 1. Treat a matrix

B

(

n \times p

) as a function that sends a vector

x

(length

p

) to a new vector

y = B x

(length

n

), where each component is a dot product:

y_{k} = j = 1 \sum p b_{k j} x_{j} .

⟳This is just equation [4] specialized to the case where the second factor is a single column (

p \to 1

Step 2. Now apply matrix

A

(

m \times n

) to

y

, producing

z = A y

z_{i} = k = 1 \sum n a_{ik} y_{k} .

⟳Same dot-product rule, now applied to the intermediate vector

y

Step 3. Substitute equation [6] into equation [7]:

z_{i} = k = 1 \sum n a_{ik} (j = 1 \sum p b_{k j} x_{j}) = j = 1 \sum p (k = 1 \sum n a_{ik} b_{k j}) x_{j} .

⟳Swap the order of summation (a finite sum lets you do this freely), then group the inner sum.

Step 4. The bracketed quantity is exactly

c_{ij} = \sum_{k} a_{ik} b_{k j}

from equation [4]. Therefore

z = (A B) x

, proving that the matrix product encodes the composition of the two transformations.

■

Arthur Cayley | 1821–1895

A British mathematician and trained lawyer, Cayley published A Memoir on the Theory of Matrices (1858), giving the first abstract treatment of matrices as objects in their own right. He defined matrix addition, multiplication, and the identity, and noted that matrix multiplication is non-commutative — a radical departure from ordinary arithmetic. Cayley also stated (and verified in small cases) what is now the Cayley–Hamilton theorem. Remarkably prolific, he produced nearly 1,000 papers, many while practicing law for 14 years before becoming Sadleirian Professor at Cambridge.

Building Intuition

Think of the dot product as a similarity score: when two vectors point in the same direction, $cos θ = 1$ and the dot product is large and positive; when perpendicular, $cos θ = 0$ and the dot product is exactly zero; when opposite, it is negative. This is why orthogonality (perpendicularity) and "zero dot product" mean the same thing.

For matrix multiplication, picture $A$ as a stack of row-detectors and $B$ as a set of column-inputs. Entry $c_{ij}$ measures how strongly row $i$ responds to column $j$ . Another lens: each column of $A B$ is a combination of the columns of $A$ , weighted by a column of $B$ . The matrix $B$ tells you how to mix the columns of $A$ .

The Dot Product in Physics

Physicists meet the dot product as work: $W = F \cdot d$ , the force times the distance in the direction of motion. Only the component of force aligned with displacement does work, which is exactly what $∥ F ∥∥ d ∥ cos θ$ captures. Push perpendicular to motion and you do zero work — the dot product is zero. This single idea reappears as power ( $P = F \cdot v$ ), magnetic flux, and the projection of one quantity onto another throughout mechanics and electromagnetism.

Worked Examples

Dot Product with a Geometric Check

Given:

a = (3, 4)

and

b = (4, - 3)

. Step 1 — Algebraic formula (eq. [2]):

a \cdot b = (3) (4) + (4) (- 3) = 12 - 12 = 0.

Step 2 — Interpret. A zero dot product means the vectors are perpendicular. Step 3 — Verify with the geometric formula (eq. [3]). Both lengths are

3^{2} + 4^{2} = 5

, so

0 = a \cdot b = 5 \cdot 5 \cdot cos θ \Rightarrow cos θ = 0 \Rightarrow θ = 9 0^{\circ} .

a \cdot b = 0, θ = 9 0^{\circ} (perpendicular)

A 2×2 Matrix Product

Given:

A = (1324), B = (5768) .

Step 1 — Check sizes.

A

2 \times 2

B

2 \times 2

; inner dimensions match, so

A B

2 \times 2

. Step 2 — Compute each entry as a dot product (eq. [4]):

$c_{11} = (1) (5) + (2) (7) = 5 + 14 = 19$
$c_{12} = (1) (6) + (2) (8) = 6 + 16 = 22$
$c_{21} = (3) (5) + (4) (7) = 15 + 28 = 43$
$c_{22} = (3) (6) + (4) (8) = 18 + 32 = 50$

A B = (19432250)

Step 3 — Confirm non-commutativity. Computing

B A

instead gives

(23313446) \neq = A B

. The order genuinely matters.

Common Misconceptions

Matrix multiplication is NOT element-wise. Beginners often try to multiply matching entries:

(A B)_{ij} \neq = a_{ij} b_{ij}

. That element-wise operation (the Hadamard product) is a different, less common thing. The real product uses rows times columns and a sum, per equation [4]. Order matters. In general

A B \neq = B A

. Sometimes only one of the two products even exists, because dimensions must line up. Never silently swap factors. The identity is not "all ones." The multiplicative identity is the matrix

I

with 1's on the diagonal and 0's elsewhere, satisfying

A I = I A = A

. A matrix of all ones does something entirely different. Claim (false): "If

A B = 0

(the zero matrix), then

A = 0

B = 0

." With ordinary numbers this is true, but matrices break it. Take

A = (1000), B = (0001) .

Then

A B = (0000),

yet neither

A

nor

B

is the zero matrix. Matrices can have zero divisors, so you cannot "cancel" factors the way you do with real numbers.

Connections

The dot product is the seed of Vector Spaces: lengths, angles, and orthogonality are all defined through it, generalizing to the abstract inner product. Matrix multiplication realizes Linear Transformations concretely — every linear map between finite spaces is "a matrix," and composing maps is multiplying matrices. Solving Systems of Linear Equations is written compactly as $A x = b$ , and the search for $x$ leads to the Matrix Inverse and the Determinant. The geometric side connects to Trigonometry through the $cos θ$ formula in equation [3].

Real-World Applications

Computer graphics & games. Rotating, scaling, and translating 3D models is done by multiplying $4 \times 4$ transformation matrices; the GPU performs millions of such products per frame.
Google's PageRank. The web's link structure is a giant matrix, and repeatedly multiplying it by a vector ranks pages by importance.
Machine learning / neural networks. Each layer of a network multiplies the input vector by a weight matrix; training adjusts those matrices.
Economics — Leontief input–output models. A matrix describes how industries consume each other's output; solving $x = A x + d$ predicts production needs.
Cryptography (Hill cipher). Messages are encoded as vectors and encrypted by multiplying with a key matrix; decryption uses its inverse.
Robotics. Chaining joint rotations (forward kinematics) multiplies a sequence of transformation matrices to locate a robot's hand.
Statistics. The normal equations $X^{⊤} X β = X^{⊤} y$ for least-squares regression are built entirely from matrix products and dot products.

Historical Development

1748 — Gabriel Cramer publishes Cramer's Rule using determinant-like arrays to solve linear systems.
1812 — Jacques Binet and Augustin-Louis Cauchy independently study products of rectangular arrays (the Binet–Cauchy identity).
1844 — Hermann Grassmann develops a general theory of "extensive quantities," anticipating vectors and bilinear products.
1850 — James Joseph Sylvester coins the word "matrix" (Latin for womb) for a rectangular array from which determinants are born.
1858 — Arthur Cayley's Memoir on the Theory of Matrices gives the first systematic algebra of matrices, defining multiplication and noting non-commutativity.
1881 — J. Willard Gibbs and Oliver Heaviside formalize vector notation, popularizing the dot product in physics and engineering.
1888 — Giuseppe Peano gives an axiomatic definition of a vector space, setting the abstract stage for these operations.
1969 — Volker Strassen shocks the field by showing matrix multiplication can be done faster than the obvious $n^{3}$ operations, opening the study of its true complexity.

Subtleties & Limits

How fast can two matrices be multiplied? The schoolbook method for

n \times n

matrices costs about

n^{3}

multiplications. Strassen (1969) lowered the exponent to

\approx 2.807

, and successive breakthroughs have pushed it down — the current record exponent

ω

sits around

2.371

(as of the early 2020s). The deep open question is whether

ω = 2

, meaning matrices could be multiplied in essentially the time it takes just to read them. No one knows; the best lower bound is only slightly above

2

, leaving a wide, famous gap that drives ongoing research in theoretical computer science.