INT8 Quantization

The Affine Quantization Map

INT8 quantization uses an affine (asymmetric) mapping between a floating-point value $r$ and its integer representation $q$. Every tensor gets its own scale $s$ and zero-point $z$.

Explanation: A real value $r$ is divided by the scale $s$, rounded to the nearest integer, shifted by the zero-point $z$, then clamped to the INT8 range [−128, 127].

Symbol guide:

• $r$ — original FP32 value
• $s$ — scale factor (FP32 scalar, one per tensor or per channel)
• $z$ — zero-point (INT8 scalar); encodes where FP32 zero lands on the integer grid
• $\lfloor \cdot \rceil$ — round-to-nearest-integer
• $\operatorname{clamp}(x, a, b)$ — clips $x$ to the range $[a, b]$

The inverse (dequantization) recovers an approximate float:

Explanation: Subtracting the zero-point and multiplying by the scale reconstructs a floating-point approximation of the original value, with quantization error at most $s/2$.

Choosing Scale and Zero-Point

Given the observed min/max of a tensor $[r_{\min},\, r_{\max}]$ and the target integer range $[q_{\min}, q_{\max}] = [-128, 127]$:

Explanation: The scale is the ratio of float range to integer range; the zero-point shifts the grid so that FP32 zero maps to an exactly representable integer, preserving sparsity-friendly zero arithmetic.

Integer Matrix Multiplication

For a linear layer $Y = XW^T$ with $X \in \mathbb{R}^{m \times k}$, $W \in \mathbb{R}^{n \times k}$, the quantized computation runs as:

Explanation: The entire inner product runs in INT32 accumulation over INT8 inputs; the two scale factors are applied only once after the full dot product, so the expensive FP32 multiply happens $k$ times fewer than in a naïve approach. The per-channel variant, which is almost always preferred for weights, assigns an independent $(s_W^{(j)}, z_W^{(j)})$ to each output channel $j$, dramatically reducing per-channel clipping error at negligible overhead.

Quantization Error

The worst-case round-trip error is bounded by half the scale:

Explanation: Reducing the float range (via outlier clipping or per-channel calibration) directly tightens this bound, which is why outlier handling is the central engineering challenge in INT8 quantization. ---

Quantization Schemes at a Glance

---

PyTorch Implementation

Post-Training Quantization (PTQ) with `torch.quantization`: ```python import torch import torch.quantization as tq

# 1. Define / load your FP32 model model_fp32 = MyModel().eval()

# 2. Fuse Conv-BN-ReLU for better quantization accuracy model_fp32 = tq.fuse_modules(model_fp32, "conv", "bn", "relu")

# 3. Insert observers to collect calibration statistics model_fp32.qconfig = tq.get_default_qconfig("fbgemm") # x86 backend tq.prepare(model_fp32, inplace=True)

# 4. Calibration pass — run representative data with torch.no_grad(): for images, _ in calibration_loader: model_fp32(images)

# 5. Convert: replace observers with fake-quant, fold scales model_int8 = tq.convert(model_fp32, inplace=False)

# 6. Verify print(model_int8) # Linear → QuantizedLinear, etc. ```

Quantization-Aware Training (QAT): ```python

# QAT inserts fake-quantization nodes during training

# so the model learns to compensate for quantization error. model_fp32.qconfig = tq.get_default_qat_qconfig("fbgemm") tq.prepare_qat(model_fp32, inplace=True)

# Train for a few more epochs with small LR optimizer = torch.optim.SGD(model_fp32.parameters(), lr=1e-4) for epoch in range(5): for images, labels in train_loader: loss = criterion(model_fp32(images), labels) optimizer.zero_grad(); loss.backward(); optimizer.step()

# Convert to true INT8 model_int8 = tq.convert(model_fp32.eval(), inplace=False) ```

Per-channel quantization for a single weight tensor (manual): ```python import torch def quantize_per_channel(W: torch.Tensor, bits: int = 8): """Symmetric per-output-channel INT8 quantization.""" qmax = 2 ** (bits - 1) - 1 # 127

# Compute per-channel max absolute value → shape [out_channels] abs_max = W.abs().amax(dim=list(range(1, W.dim())), keepdim=True) scale = abs_max / qmax # FP32 scale per channel scale = scale.clamp(min=1e-8) # avoid division by zero W_q = (W / scale).round().clamp(-qmax - 1, qmax).to(torch.int8) return W_q, scale # store scale for dequant ``` ---

---

[[Post-Training