The Pinhole Camera Model

A pinhole camera is an idealized imaging device: light from a scene passes through a single infinitesimally small aperture and lands on a flat image plane, with no lens bending the rays. Because the aperture is a single point, each scene point sends exactly one ray of light through it, so each 3D point maps to exactly one 2D point on the image plane. This gives the cleanest possible geometric model of image formation, building directly on the The Image Formation Model idea that an image is the result of geometric projection combined with radiometric measurement. The key geometric fact is similar triangles. Place the pinhole at the origin of a camera-centered coordinate system, with the optical axis along $Z$, and the image plane at distance $f$ (the focal length) in front of the aperture (a convenient fiction — physically the image forms behind the pinhole, inverted, but placing a virtual plane in front avoids the inversion and is optically equivalent). A scene point $(X, Y, Z)$, with $Z > 0$ being depth along the optical axis, projects to the image point

This is perspective projection: image coordinates scale inversely with depth, which is exactly why distant objects appear smaller and parallel lines appear to converge. The division by $Z$ is the mathematical seed of every perspective effect — foreshortening, vanishing points, and the impossibility of recovering absolute scale from a single image. To turn pixel measurements into these physical coordinates, the model separates two kinds of parameters. Intrinsic parameters describe the camera itself — focal length in pixels, the pixel aspect ratio, skew, and the principal point (where the optical axis pierces the image plane, ideally the image center). Extrinsic parameters describe where the camera sits in the world — a rotation $R$ and translation $t$ mapping world coordinates into the camera's coordinate frame. Separating these is what allows a single camera model to be reused across different poses, and it is the parameterization that Camera Calibration procedures solve for.

The projection pipeline has three logical stages, each a coordinate transformation, and each expressible as multiplication by a matrix once homogeneous coordinates are introduced.

Stage 1 — World to camera (extrinsics). A 3D point $P_w$ given in world coordinates is transformed into the camera's coordinate frame by a rigid transformation:

where $R$ is a $3\times3$ rotation matrix and $t$ is a $3\times1$ translation vector. This places the point relative to the camera, with the camera's optical axis as the $Z$-axis.

Stage 2 — Camera to normalized image plane (perspective division). The camera-frame point $(X_c, Y_c, Z_c)$ is projected through the pinhole onto a plane at unit distance:

This is the only nonlinear step in the whole pipeline — an actual division — and it is the step that discards depth information.

Stage 3 — Normalized image to pixel coordinates (intrinsics). The normalized coordinates are scaled and shifted into pixel units using the intrinsic matrix

where $f_x, f_y$ are the focal length expressed in pixels along each axis (allowing for non-square pixels), $s$ is a skew term (nearly always zero for modern sensors), and $(c_x, c_y)$ is the principal point. The trick that makes this pipeline a single matrix is homogeneous coordinates: represent the 3D point as a 4-vector $\tilde{P}_w = (X, Y, Z, 1)^T$ and the resulting 2D pixel as a 3-vector $\tilde{p} = (u, v, 1)^T$ up to scale. Then the entire projection is

where $\sim$ denotes equality up to a nonzero scalar, and $P = K[R \,|\, t]$ is the $3\times4$ camera projection matrix. The perspective division that was explicit in Stage 2 is now implicit: it is recovered by dividing the resulting 3-vector by its third coordinate. ```

# PSEUDOCODE — Pinhole projection of a 3D point to a pixel function PROJECT_POINT(P_world, R, t, K):

# P_world: 3-vector (X, Y, Z) in world coordinates

# R: 3x3 rotation matrix, t: 3x1 translation vector (extrinsics)

# K: 3x3 intrinsic matrix

# Stage 1: world -> camera frame P_cam = R * P_world + t # 3-vector (Xc, Yc, Zc) if P_cam.Z <= 0: return NULL # point is behind the camera: no image formed

# Stage 2: camera frame -> normalized image plane (perspective division) x_n = P_cam.X / P_cam.Z y_n = P_cam.Y / P_cam.Z p_n = (x_n, y_n, 1) # homogeneous normalized point

# Stage 3: normalized -> pixel coordinates via intrinsics p_pixel_h = K p_n # 3-vector (uw, v*w, w), here w = 1 u = p_pixel_h[0] / p_pixel_h[2] v = p_pixel_h[1] / p_pixel_h[2] return (u, v)

function PROJECT_MATRIX_FORM(P_world_h, P):

# P_world_h: homogeneous 4-vector (X, Y, Z, 1)

# P: 3x4 camera matrix, P = K [R | t] q = P * P_world_h # 3-vector (qx, qy, qw) if qw <= 0: return NULL # behind camera return (qx / qw, qy / qw) ``` The two functions are equivalent; `PROJECT_MATRIX_FORM` is the form used in practice because building $P$ once lets every subsequent point be projected with one $3\times4 \times 4\times1$ matrix-vector product.

The diagram makes explicit why the process is called a pipeline: the world point is first re-expressed in the camera's frame ($R, t$), then projected through the aperture with a division by depth, then rescaled onto the discrete pixel grid ($K$). Each arrow is one matrix (or one division); composing all three gives the single matrix $P$.

Time complexity. Projecting a single point is $O(1)$: two constant-size matrix-vector products ($3\times4$ times $4\times1$, then a divide). Projecting $n$ points, as when rendering a full 3D model or reprojecting a point cloud, is therefore $O(n)$, and this is optimal since every point's coordinates must be touched at least once (an $\Omega(n)$ lower bound because the output has $n$ points, each requiring examination). In practice the $3\times4$ matrix $P$ is precomputed once per camera pose, amortizing the cost of forming $K[R|t]$ (itself $O(1)$ matrix multiplications) across all $n$ points.

Space complexity. $O(1)$ additional space per camera (the $3\times4$ matrix, 12 numbers, or equivalently 5 intrinsic and 6 extrinsic degrees of freedom), plus $O(n)$ for storing the input points and output pixels, which is required simply to hold the data.

Correctness argument. The claim to verify is that the matrix formulation $\tilde p \sim K[R|t]\tilde P_w$ exactly reproduces the geometric similar-triangles projection. Write $P_c = R P_w + t = (X_c, Y_c, Z_c)^T$; by definition of homogeneous equivalence,

Dividing the first two components by the third (the homogeneous-to-Cartesian step) gives, when $s = 0$,

which is precisely "scale the similar-triangle projection $(X_c/Z_c, Y_c/Z_c)$ by focal length and shift by the principal point." This matches the Stage-2/Stage-3 derivation term for term, so the matrix form is not an approximation — it is an algebraic repackaging of the same geometry, valid whenever $Z_c \ne 0$. The invariant preserved at every stage is: the ray from the world point through the camera center, expressed in the current coordinate frame, always intersects the corresponding plane at the correctly scaled position — rotation and translation preserve this ray, and the final division reads off where it crosses the image plane.

Honesty about the model's limits. Pinhole projection is fundamentally a many-to-one map: every point along the ray from the camera center through $P$ projects to the same pixel, so depth $Z$ cannot be recovered from a single image — this is why monocular depth is ill-posed without additional priors, and why Stereo Vision uses two views to triangulate. The model also assumes an infinitesimal aperture (zero light, infinite depth of field, no blur), which is physically unrealizable; real cameras use a lens with finite aperture, introducing defocus blur and geometric distortion, refined in the study of Lenses and Optics. Finally, the model assumes a perfect, noise-free, continuous image plane, whereas real sensors discretize the plane into finite pixels, the subject of Digital Images and Sampling.

Consider a camera with intrinsics $f_x = f_y = 800$ px, principal point $(c_x, c_y) = (320, 240)$ (a $640\times480$ image), zero skew, positioned at world coordinates $t_{\text{world}} = (0, 0, -5)$ meters looking straight down the world $Z$-axis with no rotation relative to world axes ($R = I$). We want to project the world point $P_w = (0.5, 0.25, 5)$ meters — half a meter right, quarter meter up, five meters ahead of the world origin. The extrinsic transform used here is $P_c = R\,P_w + t$, with $t$ chosen as the camera position expressed as a translation applied after rotation (i.e., $t = -R\,C$ where $C$ is camera center in world coordinates; here $R = I$ and $C = (0,0,-5)$, so $t = (0,0,5)$).

So $P_w = (0.5, 0.25, 5)$ lands at pixel $(u, v) = (360, 260)$ — slightly right of and below the image center $(320, 240)$, matching intuition since the point is to the right of and above the world origin but the camera looks straight ahead (the vertical sign convention here takes image $y$ increasing downward, so "up in the world" shows as a smaller offset consistent with the chosen axis convention; the sign details depend on convention but the magnitude and mechanism are what matter). Now verify the many-to-one claim: the world point $P_w' = (1.0, 0.5, 10)$ lies on the same ray from the camera center through $P_w$ (it is $2\times$ the camera-frame vector $(0.5,0.25,10) \to (1.0, 0.5, 20)$, scaled by $2$). Projecting it:

identical normalized coordinates, hence identical pixel $(360, 260)$ despite being twice as far away. This confirms the invariant from the correctness section directly: points differing only by a positive scalar multiple along a camera-frame ray project to the same pixel, which is exactly the depth-ambiguity the pinhole model predicts. As a second check, project the camera center itself's forward axis point $(0,0,1)$ in camera frame (i.e., a point straight down the optical axis): $x_n = 0/1 = 0$, $y_n = 0/1 = 0$, giving pixel $(320, 240)$ — exactly the principal point, as required, since points on the optical axis must map to where that axis pierces the image plane.

The geometry behind the pinhole camera long predates cameras themselves: the convergence of light rays through a small aperture was documented by natural philosophers studying optics and later formalized by Renaissance artists developing linear perspective, before being rigorously connected to the mathematics of projective geometry.

:::quiz[Given camera matrix $P = K[R|t]$ and a world point in homogeneous coordinates, how is the final pixel obtained?]

• Multiply $P$ by the point and take the result directly as $(u,v)$
• Multiply $P$ by the point, then divide the first two components by the third ✓
• Add $K$ to $[R|t]$ before multiplying by the point
• Take the eigenvector of $P$ corresponding to the point

Ask the tutor: How does adding a second camera (stereo) resolve the depth ambiguity inherent in a single pinhole projection?