Object Detection: YOLO

YOLO (You Only Look Once) is a family of real-time object detection models that reframe detection as a single regression problem — predicting bounding boxes and class probabilities directly from full images in one forward pass. It matters because it broke the speed-accuracy tradeoff that plagued two-stage detectors like R-CNN, enabling detection at 45–155 FPS on standard hardware. The counterintuitive fact: YOLO is often more accurate on small-batch real-time inference than slower two-stage methods, because its global image context prevents the false positives on backgrounds that plague region-proposal pipelines.

The core insight

Instead of asking "where might objects be?" (region proposals) and then classifying each candidate, YOLO asks a single question of the entire image at once: "For every cell in this grid, what objects are centered here, how big are they, and what class are they?" The network divides the image into an S×S grid, and each cell simultaneously predicts B bounding boxes with confidence scores and C class probabilities — all in a single forward pass. This unified architecture means gradients flow through the entire detection pipeline end-to-end, and the model learns a richer global representation. The "aha moment" is realizing that object detection is just a structured regression problem dressed up in grid clothing.

How It Works

Grid Division and Prediction Tensor

Given an input image of size $W \times H$ , YOLO divides it into an $S \times S$ grid. Each grid cell predicts $B$ bounding boxes and $C$ conditional class probabilities. The full output tensor has shape:

\quad \text{Output} \in \mathbb{R}^{S \times S \times (B \cdot 5 + C)}

Explanation: Each of the S² grid cells independently emits B boxes (each parameterized by 5 values: x, y, w, h, confidence) plus C class scores, so the entire detection problem collapses to predicting this single dense tensor. Where:

$S$ — grid size (e.g., 7 in YOLOv1, 13/26/52 in YOLOv3)
$B$ — number of anchor boxes per cell (e.g., 2 in YOLOv1, 3 in YOLOv3)
$C$ — number of object classes
The 5 values per box: $(t_x, t_y, t_w, t_h, \text{conf})$

Bounding Box Encoding (YOLOv2+)

Rather than predicting absolute coordinates, YOLO predicts offsets relative to anchor priors. Given an anchor of width $p_w$ and height $p_h$ , and the cell's top-left corner at $(c_x, c_y)$ :

\quad b_x = \sigma(t_x) + c_x, \quad b_y = \sigma(t_y) + c_y, \quad b_w = p_w e^{t_w}, \quad b_h = p_h e^{t_h}

Explanation: The sigmoid constrains the predicted center offset to stay within the responsible grid cell (preventing training instability), while the exponential maps unbounded network outputs to positive box dimensions scaled by the anchor prior. Where:

$\sigma$ — sigmoid function, squashes $t_x, t_y \in (-\infty, \infty)$ to $(0, 1)$
$t_x, t_y$ — raw network outputs for center offset
$t_w, t_h$ — raw network outputs for width/height log-scale adjustment
$c_x, c_y$ — integer grid cell offsets (column, row)
$p_w, p_h$ — anchor prior dimensions (from k-means clustering on training boxes)

Objectness Confidence

Each box prediction includes an objectness score — the probability that an object exists there, weighted by how well the box fits:

\quad \text{Conf} = \Pr(\text{Object}) \times \text{IoU}(\text{pred}, \text{gt})

Explanation: During training the confidence target is set to the IoU between the predicted box and the ground-truth box, so the network learns to predict not just "is something here" but also "how well am I boxing it." Where:

$\Pr(\text{Object})$ — 1 if any ground-truth box's center falls in this cell, 0 otherwise
$\text{IoU}(\text{pred}, \text{gt})$ — Intersection over Union between predicted and ground-truth boxes

Multi-Part Loss Function (YOLOv1)

The full YOLO training loss is a weighted sum over localization, confidence, and classification terms:

\quad \mathcal{L} = \lambda_\text{coord} \sum_{i,j} \mathbf{1}_{ij}^\text{obj}\!\left[(x_i - \hat{x}_i)^2 + (y_i - \hat{y}_i)^2 + (\sqrt{w_i} - \sqrt{\hat{w}_i})^2 + (\sqrt{h_i} - \sqrt{\hat{h}_i})^2\right] + \sum_{i,j} \mathbf{1}_{ij}^\text{obj}(C_i - \hat{C}_i)^2 + \lambda_\text{noobj}\sum_{i,j}\mathbf{1}_{ij}^\text{noobj}(C_i - \hat{C}_i)^2 + \sum_{i} \mathbf{1}_i^\text{obj}\sum_{c \in \text{classes}}(p_i(c) - \hat{p}_i(c))^2

Explanation: The square root on width and height deemphasizes large errors in big boxes (a 10px error on a 200px box matters less than on a 20px box), while λ_noobj downweights the overwhelming majority of empty cells so they don't swamp the gradient signal from real detections. Where:

$\lambda_\text{coord} = 5$ — upweights localization loss to prioritize box accuracy
$\lambda_\text{noobj} = 0.5$ — downweights confidence loss for empty cells
$\mathbf{1}_{ij}^\text{obj}$ — indicator: 1 if object $j$ in cell $i$ is the "responsible" predictor
$\hat{x}, \hat{y}, \hat{w}, \hat{h}$ — ground-truth box coordinates
$C_i, \hat{C}_i$ — predicted and target confidence scores
$p_i(c), \hat{p}_i(c)$ — predicted and target class probabilities

IoU (Intersection over Union)

\quad \text{IoU}(A, B) = \frac{|A \cap B|}{|A \cup B|} = \frac{|A \cap B|}{|A| + |B| - |A \cap B|}

Explanation: IoU is scale-invariant — a 0.5 IoU means the overlap is half the combined area regardless of whether the boxes are 10px or 1000px, making it a consistent measure of localization quality across object sizes. Where:

$A$ — predicted bounding box region
$B$ — ground-truth bounding box region
$|A \cap B|$ — pixel area of overlap
$|A \cup B|$ — pixel area of union

PyTorch: YOLO Head Forward Pass

```python import torch import torch.nn as nn class YOLOHead(nn.Module): """Single YOLO detection head for one scale.""" def __init__(self, in_channels: int, num_anchors: int, num_classes: int): super().__init__() self.num_anchors = num_anchors self.num_classes = num_classes out_channels = num_anchors (5 + num_classes) self.conv = nn.Conv2d(in_channels, out_channels, kernel_size=1) def forward(self, x: torch.Tensor, anchors: torch.Tensor): """ x: (B, C, S, S) — feature map from backbone anchors: (num_anchors, 2) — prior (pw, ph) for this scale returns: (B, S, S, num_anchors, 5 + num_classes) decoded predictions """ B, _, S, _ = x.shape raw = self.conv(x) # (B, num_anchors(5+C), S, S)

# Reshape to (B, S, S, num_anchors, 5+C) raw = raw.permute(0, 2, 3, 1).contiguous() raw = raw.view(B, S, S, self.num_anchors, 5 + self.num_classes)

# Build grid offsets cx, cy shape: (1, S, S, 1) grid_y, grid_x = torch.meshgrid( torch.arange(S, dtype=torch.float32), torch.arange(S, dtype=torch.float32), indexing='ij' ) cx = grid_x.unsqueeze(0).unsqueeze(-1) # (1, S, S, 1) cy = grid_y.unsqueeze(0).unsqueeze(-1)

# Decode tx,ty → bx,by via sigmoid + offset (Eq. [2]) bx = (torch.sigmoid(raw[..., 0]) + cx) / S by = (torch.sigmoid(raw[..., 1]) + cy) / S

# Decode tw,th → bw,bh via exp anchor prior pw = anchors[:, 0].view(1, 1, 1, -1) ph = anchors[:, 1].view(1, 1, 1, -1) bw = pw torch.exp(raw[..., 2]) bh = ph * torch.exp(raw[..., 3]) conf = torch.sigmoid(raw[..., 4]) # objectness cls = torch.sigmoid(raw[..., 5:]) # class probs (BCE targets) return torch.stack([bx, by, bw, bh], dim=-1), conf, cls ```

researcher · Joseph Redmon | b. 1987 | Inventor of YOLO

Joseph Redmon, a computer science PhD graduate from the University of Washington, introduced YOLO in 2015 with his advisor Ali Farhadi, framing object detection as a single regression problem for the first time. His contributions spanned YOLOv1 through YOLOv3, each dramatically improving speed and accuracy over prior two-stage methods like Faster R-CNN. In a surprising turn, Redmon publicly quit computer vision research in 2020 citing concerns about military applications and surveillance uses of his work — one of the few prominent researchers to step away from a field he largely defined.

researcher · Ali Farhadi | b. 1978 | Co-inventor of YOLO and Darknet

Ali Farhadi is a professor at the University of Washington and co-creator of YOLO and the Darknet framework, which provided the lightweight C/CUDA backbone that made real-time detection feasible on consumer GPUs. He led the team that demonstrated YOLO could process video at 155 FPS with the Fast YOLO variant — an order of magnitude faster than contemporaneous detectors. Farhadi later co-founded the Allen Institute for AI's vision group and Apple's AI research arm hired him, illustrating how foundational the YOLO work became to industrial computer vision.

Figure

YOLOv3 Multi-Scale Detection Architecture

Figure

YOLO Grid Cell Prediction Detail