Floating Point Arithmetic and IEEE 754

Floating point arithmetic represents real numbers in a finite computer word using a sign, a scaled significand, and an exponent — trading exact representation for enormous dynamic range. The IEEE 754 standard (1985, revised 2008/2019) fixes the bit layouts, rounding rules, and special values (±0, ±∞, NaN) used by essentially every modern processor. It applies everywhere numbers are computed: graphics, simulation, finance, machine learning, embedded control. Key fact: most decimal fractions (even 0.1) are not exactly representable in binary floating point.

A floating point number approximates a real value $x$ as $\pm m \times β^{e}$ , where $m$ is a fixed-precision significand, $β$ is the radix (2 in IEEE 754), and $e$ is a bounded integer exponent. This format matters because it lets a 32- or 64-bit word represent magnitudes from subatomic to cosmological scales while keeping relative error roughly constant. Before IEEE 754, every vendor used a different format, so the same program gave different answers on different machines. The standard imposed a single, carefully specified model — including correctly rounded basic operations — that made numerical code portable and reproducible. The surprising fact: although floating point feels like "the real numbers," it is a finite, unevenly-spaced grid where addition is not associative and exact arithmetic laws routinely fail.

The Core Insight

Floating point keeps a constant number of significant digits, not a constant absolute precision. Because the exponent slides the binary point, the spacing between representable numbers grows with magnitude — the gaps double each time you cross a power of two. The fundamental guarantee is therefore not "small absolute error" but "small relative error": every real number in range is approximated to within a fixed fraction $u$ (the unit roundoff). All of numerical analysis flows from this single bound.

Formal Statement

Normalized Floating Point Number

A floating point number system $F (β, p, e_{m i n}, e_{m a x})$ consists of numbers of the form

x = (- 1)^{s} \times (d_{0} . d_{1} d_{2} \dots d_{p - 1})_{β} \times β^{e}

where:

$s \in {0, 1}$ is the sign bit,
$β$ is the radix (base), $β = 2$ in IEEE 754,
$p$ is the precision (number of significand digits $d_{i}$ , with $0 \leq d_{i} < β$ ),
$e$ is the exponent, $e_{m i n} \leq e \leq e_{m a x}$ ,
the value is normalized when $d_{0} \neq = 0$ (for $β = 2$ , this forces $d_{0} = 1$ ).

⟳the significand

1. d_{1} \dots d_{p - 1}

lies in

[1, 2)

; multiplying by

2^{e}

rescales it to any binade.

IEEE 754 Bit Encoding

A binary IEEE 754 value packs three fields into a word: sign $S$ (1 bit), biased exponent $E$ (width $w$ bits), and trailing significand $T$ (width $t = p - 1$ bits). The stored exponent is biased: $E = e + bias$ , with $bias = 2^{w - 1} - 1$ . The decoded value for a normal number is

x = (- 1)^{S} \times (1 + T \cdot 2^{- t}) \times 2^{E - bias}

The leading 1 is the hidden bit — not stored, recovered for free in binary.

⟳

T

is the integer formed by the fraction bits; dividing by

2^{t}

places the binary point just after the implicit leading 1.

| Format | total bits | $w$ (exp) | $t$ (frac) | $p$ | bias | $u = 2^{- p}$ | |---|---|---|---|---|---|---| | binary16 (half) | 16 | 5 | 10 | 11 | 15 | $2^{- 11}$ | | binary32 (single) | 32 | 8 | 23 | 24 | 127 | $2^{- 24}$ | | binary64 (double) | 64 | 11 | 52 | 53 | 1023 | $2^{- 53}$ | | binary128 (quad) | 128 | 15 | 112 | 113 | 16383 | $2^{- 113}$ |

Special Encodings

The extreme exponent fields are reserved:

$E = 0$ , $T = 0$ → signed zero $\pm 0$ .
$E = 0$ , $T \neq = 0$ → subnormal (denormal): value $(- 1)^{S} (0 + T 2^{- t}) 2^{1 - bias}$ , with implicit leading 0, filling the gap around zero (gradual underflow).
$E = 2^{w} - 1$ , $T = 0$ → infinity $\pm \infty$ .
$E = 2^{w} - 1$ , $T \neq = 0$ → NaN (Not a Number); quiet vs. signaling distinguished by the top fraction bit.

Rounding Error Bound

Let $fl (x)$ denote the correctly rounded value of a real $x$ in the normal range of $F$ . Then there exists $δ$ with

fl (x) = x (1 + δ), ∣ δ ∣ \leq u = \frac{1}{2} β^{1 - p} = 2^{- p}

where $u$ is the unit roundoff. Moreover, for any IEEE 754 basic operation $op \in {+, -, \times, \div,}$ ,

fl (a op b) = (a op b) (1 + δ), ∣ δ ∣ \leq u

⟳round-to-nearest selects the closest grid point; the worst case is the midpoint, half a unit in the last place (ulp) away — divide that gap by the magnitude

\geq β^{e}

Why It Holds / Derivation

Of the relative error bound (3)

Step 1. Fix

x \neq = 0

in the normal range. Write

∣ x ∣ = m \cdot β^{e}

with

m \in [1, β)

. The representable numbers in this binade are spaced exactly

β^{e - (p - 1)}

apart; this spacing is one ulp. Step 2. Round-to-nearest maps

x

to the closest grid point, so the absolute error satisfies

∣ fl (x) - x ∣ \leq \frac{1}{2} ulp = \frac{1}{2} β^{e - (p - 1)} = \frac{1}{2} β^{e} β^{1 - p} .

Step 3. Since the significand

m \geq 1

, we have

∣ x ∣ = m β^{e} \geq β^{e}

. Divide:

\frac{∣ fl ( x ) - x ∣}{∣ x ∣} \leq \frac{\frac{1}{2} β ^{e} β ^{1 - p}}{β ^{e}} = \frac{1}{2} β^{1 - p} = u .

Step 4. Define

δ = (fl (x) - x) / x

; then

∣ δ ∣ \leq u

and

fl (x) = x (1 + δ)

. For

β = 2

u = 2^{- p}

. Step 5. Equation (4) follows from the IEEE 754 mandate that each basic operation behave as if computed to infinite precision and then rounded once. Applying (3) to the exact result

a op b

gives the bound.

■

The uneven grid of floating point: spacing doubles at each power of two

William Kahan | b. 1933

Often called "the Father of Floating Point," Kahan led the design of the Intel 8087 math coprocessor arithmetic that became the template for IEEE 754. A Berkeley numerical analyst, he championed correct rounding, gradual underflow (subnormals), and the principled handling of exceptions — features that vendors initially resisted as too costly. His compensated summation algorithm and tireless advocacy for reproducible arithmetic earned him the 1989 Turing Award. Kahan's enduring lesson: the hard part of floating point is not the average case but the adversarial corner cases that silently corrupt results.

Building Intuition

Think of floating point as scientific notation with a fixed budget of significant figures. You always carry, say, 16 decimal digits — whether the number is $6.022 \times 1 0^{23}$ or $1.602 \times 1 0^{- 19}$ . The exponent is "free" range; the significand is where precision lives. So the number line is not a smooth continuum but a logarithmically-zoned ruler: fine ticks near zero, coarsening as you go out. Two consequences become obvious. First, adding a tiny number to a huge one can do nothing — the tiny one falls below the local tick spacing and rounds away. Second, subtracting two nearly-equal numbers wipes out leading digits, leaving noise (catastrophic cancellation). The constant-relative-error picture is the master key: it tells you errors compound multiplicatively, and that you should fear differences, not sums.

Worked Examples

Encoding 0.1 in binary64 — why it's inexact

Given: Decimal

0.1

. Find its binary64 representation and the rounding error. Step 1. In binary,

0.1 = 0.0001100110011 \dots_{2}

, a repeating fraction (period "0011"). It is not a finite binary fraction, so it cannot be exact. Step 2. Normalize:

0.1 = 1.100110011 \dots_{2} \times 2^{- 4}

, so

e = - 4

, biased

E = - 4 + 1023 = 1019 = 01111111011_{2}

. Step 3. The 52-bit fraction is the repeating pattern truncated and rounded-to-nearest:

T \cdot 2^{- 52} = 0.1001100110011 \dots (rounded) .

The stored value is the nearest double:

fl (0.1) = 0.1000000000000000055511151231257827021181583404541015625.

Step 4. The error is

fl (0.1) - 0.1 \approx + 5.55 \times 1 0^{- 18}, \frac{∣ δ ∣}{0.1} \approx 5.6 \times 1 0^{- 17} < u = 2^{- 53} \approx 1.1 \times 1 0^{- 16} .

fl (0.1) \approx 0.1 + 5.55 \times 1 0^{- 18}, ∣ δ ∣ \leq 2^{- 53}

This is exactly why 0.1 + 0.2 != 0.3 in most languages.

Catastrophic cancellation in a quadratic root

Given: Solve

x^{2} - 200000 x + 1 = 0

in binary64 using the textbook formula. Here

a = 1

b = - 200000

c = 1

. Step 1. Discriminant:

Δ = b^{2} - 4 a c = 4 \times 1 0^{10} - 4 = 39999999996

. Then

Δ \approx 199999.99999000

. Step 2. The "stable" root:

x_{1} = \frac{- b + Δ}{2} = \frac{200000 + 199999.99999}{2} \approx 199999.999995

. Fine. Step 3. The "naive" small root:

x_{2} = \frac{- b - Δ}{2} = \frac{200000 - 199999.99999000}{2} .

Here two ~

2 \times 1 0^{5}

numbers nearly cancel: about 11 leading digits agree and vanish, leaving only a few accurate digits. Step 4. Stable alternative via

x_{1} x_{2} = c / a = 1

x_{2} = \frac{c}{a x _{1}} = \frac{1}{199999.999995} = 5.0000000125 \times 1 0^{- 6} .

x_{2} \approx 5.0000000125 \times 1 0^{- 6} (via c / (a x_{1})), accurate to full precision

The subtraction form loses ~11 significant digits; the product form keeps all of them.

Common Misconceptions

Floating point is not "approximate real arithmetic with random noise." It is deterministic and fully specified: every IEEE 754 operation produces a unique, correctly rounded result, identical across conforming hardware. The "errors" are not noise — they are exact, reproducible consequences of rounding. Two further traps:

Associativity fails: $(a + b) + c \neq = a + (b + c)$ in general, so compilers may not freely reorder sums, and parallel reductions can give different (still valid) answers.
Equality testing is dangerous: never write if (x == 0.1) after computing x; compare with a tolerance instead.

Addition is not associative

In binary64, let $a = 1 0^{16}$ , $b = 1$ , $c = - 1 0^{16}$ .

Compute left-to-right: $(a + b) + c$ . Since $1 < ulp (1 0^{16}) = 2$ , the sum $a + b$ rounds back to $1 0^{16}$ ; then $+ c$ gives $0$ .

Compute right-grouped: $a + (b + c) = 1 0^{16} + (1 - 1 0^{16}) = 1 0^{16} + (- 9999999999999999) = 1$ .

Same three numbers, two different answers — associativity, an axiom of real arithmetic, simply does not hold.

Connections

Floating point sits at the crossroads of pure and applied mathematics. The error bound (3) is the entry point to Numerical Stability and Conditioning, where the condition number measures sensitivity of a problem and backward error measures the quality of an algorithm. The base- $β$ positional encoding generalizes Positional Number Systems and connects to Binary Representation of Integers for the exponent and significand fields. Rounding analysis underpins Numerical Linear Algebra (e.g., the stability of Gaussian elimination) and Numerical Solution of ODEs. The logarithmic spacing of the grid is essentially a discretized Logarithm, which is why relative error, not absolute error, is the natural metric.

Real-World Applications

Graphics & GPUs — binary32 (and increasingly binary16/bfloat16) drives shading, geometry, and ray tracing; the depth buffer's nonuniform precision is a direct artifact of floating point spacing.
Machine learning — training uses mixed precision (bfloat16, float16, float8); the entire scaling-law economics of large models hinges on low-precision floating point throughput.
Scientific simulation — climate models, CFD, and molecular dynamics rely on binary64; error analysis determines time-step stability and conservation drift.
Finance — currency is not stored as floating binary (because 0.10 is inexact); regulations mandate decimal arithmetic (IEEE 754 decimal64/decimal128) to avoid sub-cent rounding scandals.
GPS & navigation — position differencing requires careful avoidance of cancellation; double precision and reformulation keep meter-level accuracy from $1 0^{7}$ -meter coordinates.
Spacecraft control — the Ariane 5 Flight 501 explosion (1996) was a floating-to-integer overflow conversion error, a landmark cautionary tale.
Reproducible computing — IEEE 754 correct rounding lets regulatory and scientific results be bit-for-bit reproduced across machines, essential for auditing and peer review.

Historical Development

1914 — Leonardo Torres y Quevedo describes floating point in an electromechanical computing essay.
1941 — Konrad Zuse's Z3 uses a binary floating point format with exponent and significand — arguably the first working implementation.
1950s–1970s — Every vendor (IBM 360 hex floats, CDC, Cray, DEC) invents its own format; precision, rounding, and range all differ, making portable numerics nearly impossible.
1976 — Intel begins the 8087 coprocessor project; Kahan and students are hired to design its arithmetic.
1977–1985 — A fierce standards committee (the "KCS" draft by Kahan, Coonen, Stone) debates gradual underflow, NaNs, and directed rounding.
1985 — IEEE 754-1985 is ratified: formats, rounding modes, exceptions, and special values become law.
1987 — IEEE 854 generalizes to any radix (covering decimal).
2008 — IEEE 754-2008 merges 754/854, adds binary16, decimal formats, and the fused multiply-add (FMA).
2019 — IEEE 754-2019 refines recommended functions, augmented operations, and minimum/maximum semantics.

Subtleties & Limits

Several deep difficulties remain practical headaches rather than solved theory:

The Table-Maker's Dilemma: correctly rounding transcendental functions ( $sin$ , $exp$ , $lo g$ ) requires knowing how many extra digits suffice to decide the rounding direction — and no general bound is known. Implementations use heuristics (Ziv's strategy), so correctly rounded libm functions are still not universal.
Reproducibility vs. performance: FMA, vectorization, and parallel reductions all change rounding; bit-reproducible parallel summation is an active research and engineering area.
Low-precision proliferation: float8, bfloat16, and posit/unum alternatives challenge IEEE 754's dominance; whether a successor standard should replace fixed-exponent floats with tapered precision is genuinely open.
Verified numerics: automatically proving tight error bounds for whole programs (not single operations) — interval and affine arithmetic help but remain conservative.