Imaging Systems and Optical Transfer Functions

An imaging system fuses the machinery of two earlier chapters into one astonishingly powerful statement. From Fourier analysis it borrows the convolution theorem — that a linear, shift-invariant operation becomes multiplication once you look in the frequency domain. From Fraunhofer diffraction it borrows the fact that the field a lens delivers to its focus is the Fourier transform of the field across its aperture. Put those together and every lens, microscope, telescope and camera becomes the same object: a spatial-frequency filter. The image of a mathematical point is never a point, because a finite aperture cannot collect the steeply tilted plane waves that fine detail radiates; it is a blurred spot called the point-spread function (PSF). If the blur is the same everywhere in the field — the isoplanatic assumption — the image is the geometric image convolved with that PSF. The frequency-domain view is where the subject earns its name. The Fourier transform of the PSF is the optical transfer function (OTF); its modulus is the modulation transfer function (MTF), which tells you precisely how much contrast a sinusoidal test pattern of a given fineness survives on its way through the instrument. One theorem organizes everything: the OTF is the normalized autocorrelation of the pupil function. From it drop the diffraction-limited cutoff frequency, the Airy pattern, the resolution criteria of Rayleigh, Abbe and Sparrow, and the exact way aberrations erode contrast without buying resolution. Diffraction builds the outer wall; aberrations only decide how much light survives inside it — the working language of lens design, super-resolution microscopy, and the lithography that prints every microchip.

Key takeaways
  • An imaging system maps object intensity (incoherent light) or object field (coherent light) to the image plane by convolution with the point-spread function — the image of a single ideal point source.
  • The optical transfer function is the Fourier transform of the PSF; its modulus, the MTF, is the contrast-transfer curve versus spatial frequency, and its phase, the PTF, records contrast reversals.
  • For a diffraction-limited circular aperture the PSF is the Airy pattern and the incoherent MTF is a closed form that cuts off cleanly at fc=2 NA/λ=1/(λF#)f_c = 2\,\mathrm{NA}/\lambda = 1/(\lambda F_\#).
  • The OTF equals the normalized autocorrelation of the pupil, so aberrations (pupil phase) and apodization reshape the PSF and lower the MTF without ever moving the cutoff.
  • Coherent imaging is linear in complex field with cutoff NA/λ\mathrm{NA}/\lambda; incoherent imaging is linear in intensity with a cutoff twice as large — the two are not directly comparable.
  • Key idea: imaging is linear, shift-invariant filtering — the image spectrum equals the object spectrum times a transfer function that is the autocorrelation of the pupil; diffraction fixes the cutoff, aberrations lower the MTF inside it.
The core insight

Imaging is linear, shift-invariant filtering: the image spectrum equals the object spectrum times a transfer function that is the autocorrelation of the pupil. Diffraction fixes the cutoff frequency; aberrations only lower the MTF inside it.

Where this lives today

The transfer-function picture is the daily tool of anyone who specifies an imaging instrument. Camera and lens makers publish MTF charts — contrast at 10 and 30 lp/mm versus field position — and a system's MTF is the product of the lens MTF, the sensor's pixel-aperture sinc, and any anti-alias filter. Semiconductor lithography lives right at the Abbe limit, printing half-pitch k1λ/NAk_1\lambda/\mathrm{NA}: water immersion raised the effective NA past 1.35 and phase-shift masks engineer the object and pupil spectra to shrink k1k_1, before extreme-ultraviolet sources cut λ\lambda to 13.5 nm. In astronomy, atmospheric turbulence stamps a random phase across the pupil and collapses the long-exposure MTF to that of a coherence patch of size r0r_0; adaptive optics cancels that pupil phase hundreds of times a second, restoring the diffraction-limited MTF of the full aperture — a real-time application of the Strehl and pupil-phase formalism. The most striking modern twist is that the Abbe limit binds only linear far-field imaging. Structured-illumination microscopy beats the sample against a known grating to fold high object frequencies down into the passband, doubling the effective cutoff; STED and single-molecule methods (PALM, STORM) exploit nonlinear fluorescence and localization to place emitters to a few nanometres. None violate the pupil autocorrelation — they add information the linear OTF cannot carry, so the transfer function still states exactly what one linear snapshot can and cannot see.

The physical picture

Treat the lens aperture as a bandpass filter for spatial frequencies. Fine object detail radiates steeply tilted plane waves; only tilts that still fit through the aperture are collected and re-focused. Detail finer than the cutoff radiates waves that miss the lens entirely, so no refocusing recovers it — only a wider aperture (higher NA) or a shorter wavelength collects those waves. This is why resolution is a property of the aperture, not the magnification: blowing the image up larger only magnifies the blur. The MTF is best read as a contrast-survival curve. Photograph a ladder of black-and-white bar targets of increasing spatial frequency: coarse bars come through crisp with contrast near unity, fine bars come through faded, and beyond the cutoff they wash out to uniform grey. The whole curve — not a single resolution number — characterizes the instrument, which is why lens makers publish MTF rather than a lines-per-millimetre figure.

Figure
The diffraction-limited MTF of a circular aperture: contrast is near unity for coarse detail and fades to zero exactly at the cutoff f_c = 2·NA/λ, passing through MTF = 0.391 at half the cutoff. The coherent transfer (flat, cutoff at ½ f_c) is not directly comparable — it is linear in field, not intensity
Contrast-survival curve: incoherent MTF versus normalized spatial frequency 0 0.25 0.5 0.75 1 0 0.25 0.5 0.75 1 normalized spatial frequency f / f_c transferred contrast MTF coherent |ATF| (flat, cutoff at ½ f_c) incoherent MTF MTF(½ f_c) = 0.391 cutoff f_c = 2·NA/λ

The curve has exactly this shape because of geometry. The pupil-autocorrelation theorem says the OTF at frequency ff equals the fractional overlap area of the pupil with a copy of itself shifted by s=λzifs = \lambda z_i f. Push to higher frequency and the disks slide apart, the overlap shrinks, and when the shift reaches a full pupil diameter they separate completely — that shift is the cutoff. Aberrations enter as wrinkles in the pupil phase: autocorrelating a wrinkled pupil with its shifted self lets the phases partly cancel, so the overlap integral drops, the MTF sags, and it can even go negative (contrast reversal). But the boundary where the disks separate never moves. Aberrations spend contrast; they cannot buy resolution.

Figure
The optical transfer function is the normalized autocorrelation of the pupil — the overlap area of the pupil with a copy shifted by s = λ·z_i·f, over the pupil area. At shift s = ρ₀ the overlap gives OTF = 0.391; at s = 2ρ₀ the disks separate and OTF = 0, the cutoff f_c
OTF(f) = pupil ∩ shifted pupil, normalized by pupil area shift s = ρ₀ (f = ½ f_c) s overlap → OTF = 0.391 ρ₀ = pupil radius shift s = 2ρ₀ (f = f_c) s = 2ρ₀ no overlap → OTF = 0 cutoff f_c = 2·NA/λ

The core equations

Start from the operational statement of imaging. For an incoherent object — one whose points radiate with no fixed phase relationship, covering most self-luminous and diffusely-lit scenes — intensities add, and the shift-invariant instrument reproduces every object point as the same blur hh.

gi(x,y)=∬h(x−x′, y−y′) go(x′,y′) dx′ dy′=(h∗go)(x,y)(1)g_i(x,y)=\iint h(x-x',\,y-y')\,g_o(x',y')\,\mathrm{d}x'\,\mathrm{d}y' = (h * g_o)(x,y) \qquad (1)

where: gig_i — image-plane intensity; gog_o — geometric (magnification-scaled) image intensity; hh — intensity point-spread function, the image of a point; ∗* — 2-D convolution. The image is the object smeared by a fixed blur kernel — the sharper the kernel, the sharper the image. The convolution theorem converts the smear into a product. Fourier-transforming both sides replaces hh by the OTF and the object by its spatial-frequency spectrum; the modulus and phase of the OTF carry distinct information and get their own names.

H(fx,fy)=F{h(x,y)},MTF=∣H∣H(0,0),PTF=arg⁡H(2)H(f_x,f_y)=\mathcal{F}\{h(x,y)\},\qquad \mathrm{MTF}=\frac{|H|}{H(0,0)},\qquad \mathrm{PTF}=\arg H \qquad (2)

where: HH — optical transfer function, normalized so H(0,0)H(0,0) is the total collected energy; fx,fyf_x,f_y — spatial frequencies (cycles per unit length in the image plane); MTF\mathrm{MTF} — fraction of contrast transferred; PTF\mathrm{PTF} — phase, recording lateral shifts and sign reversals. The MTF answers "how much contrast survives," the PTF "does the pattern move or invert?" Both the PSF and the coherent transfer come from the aperture. Fraunhofer diffraction makes the amplitude PSF the Fourier transform of the pupil, so the intensity PSF is its squared modulus; for coherent light the system is linear in field and the amplitude transfer function is the pupil itself, rescaled so pupil position maps to spatial frequency.

h(x,y)∝∣F{P(u,v)}∣2,ATF(fx,fy)=P(λzifx, λzify)(3)h(x,y)\propto\left|\mathcal{F}\{P(u,v)\}\right|^2,\qquad \mathrm{ATF}(f_x,f_y)=P(\lambda z_i f_x,\ \lambda z_i f_y) \qquad (3)

where: P(u,v)P(u,v) — pupil function, 1 inside the aperture and 0 outside, carrying phase eikWe^{ikW} for a wavefront aberration WW; ATF\mathrm{ATF} — coherent (amplitude) transfer function, equal to the scaled pupil; ziz_i — exit-pupil-to-image distance; k=2π/λk=2\pi/\lambda. The aperture is literally the coherent filter, hard-edged at the rim. The central theorem follows: because the incoherent PSF is a squared modulus, its transform is an autocorrelation, and unwinding the transforms shows the OTF is the pupil autocorrelated with itself — the equation the two figures drew.

H(fx,fy)=∬P ⁣(u+λzifx2, v+λzify2)P∗ ⁣(u−λzifx2, v−λzify2)du dv∬∣P(u,v)∣2 du dv(4)H(f_x,f_y)=\dfrac{\displaystyle\iint P\!\left(u+\tfrac{\lambda z_i f_x}{2},\,v+\tfrac{\lambda z_i f_y}{2}\right)P^{*}\!\left(u-\tfrac{\lambda z_i f_x}{2},\,v-\tfrac{\lambda z_i f_y}{2}\right)\mathrm{d}u\,\mathrm{d}v}{\displaystyle\iint |P(u,v)|^2\,\mathrm{d}u\,\mathrm{d}v} \qquad (4)

where: P∗P^{*} — conjugate pupil, carrying the opposite aberration phase −W-W; λzif\lambda z_i f — pupil-plane shift for spatial frequency ff. The transferred contrast at frequency ff is the fractional overlap of the pupil with a copy shifted by λzif\lambda z_i f — overlap area over total area. Because the pupil has finite radius ρ0\rho_0, the coherent transfer stops when λzif\lambda z_i f reaches ρ0\rho_0, while the autocorrelation spans twice the pupil radius and stops only at 2ρ02\rho_0. That factor of two is the single most important number-fact of the subject.

fcinc=2 NAλ=1λF#,fccoh=NAλ=12λF#(5)f_c^{\mathrm{inc}}=\dfrac{2\,\mathrm{NA}}{\lambda}=\dfrac{1}{\lambda F_\#},\qquad f_c^{\mathrm{coh}}=\dfrac{\mathrm{NA}}{\lambda}=\dfrac{1}{2\lambda F_\#} \qquad (5)

where: NA=nsin⁡θmax⁡\mathrm{NA}=n\sin\theta_{\max} — image-space numerical aperture; F#=f/D≈1/(2 NA)F_\# = f/D\approx 1/(2\,\mathrm{NA}) in air — f-number; λ\lambda — wavelength in the image-space medium. The incoherent system reaches twice the coherent cutoff, but with graded not flat contrast — "twice the cutoff" is not "twice as good." For the circular aperture the coherent PSF is the Airy pattern — the squared jinc function — whose first dark ring sets the classical resolution scale, and placing one Airy peak on a neighbour's first zero defines the Rayleigh criterion.

h(r)=[2J1 ⁣(πr/(λF#))πr/(λF#)]2,rAiry=1.22 λF#=0.61 λNA(6)h(r)=\left[\dfrac{2J_1\!\left(\pi r/(\lambda F_\#)\right)}{\pi r/(\lambda F_\#)}\right]^2,\qquad r_{\mathrm{Airy}}=1.22\,\lambda F_\# = \dfrac{0.61\,\lambda}{\mathrm{NA}} \qquad (6)

where: J1J_1 — first-order Bessel function; rr — radial image-plane coordinate; rAiryr_{\mathrm{Airy}} — first-dark-ring (Rayleigh) radius. About 83.8% of the point's energy lands inside this ring (the Airy disk), and two points closer than rAiryr_{\mathrm{Airy}} blur into one. The competing conventions differ only by a constant: Rayleigh 0.61 λ/NA0.61\,\lambda/\mathrm{NA}, Abbe λ/(2 NA)=1/fcinc\lambda/(2\,\mathrm{NA})=1/f_c^{\mathrm{inc}}, and Sparrow ≈0.47 λ/NA\approx0.47\,\lambda/\mathrm{NA}.

Derivation AThe optical transfer function as the pupil autocorrelation

The goal of this derivation is to prove the theorem the whole article rests on — that the incoherent optical transfer function is the normalized autocorrelation of the pupil. We start from the diffraction fact that the amplitude PSF is the Fourier transform of the pupil, use the property that the transform of a squared modulus is an autocorrelation, and undo one nested transform so the pupil reappears correlated with itself. The payoff is to convert a hard question — what is a lens's transfer function? — into a picture of two shifted disks sliding across each other, from which the cutoff, the roll-off and the effect of aberrations all follow by inspection.

Step 1. Write the coherent amplitude PSF as a(x,y)=F{P(u,v)}a(x,y)=\mathcal{F}\{P(u,v)\}, so the incoherent intensity PSF is h=∣a∣2h=|a|^2. Why: an incoherent object adds intensities not fields, so the relevant blur kernel is the modulus-squared of the coherent response — exactly what the detector records.

Step 2. Transform the PSF and apply the autocorrelation theorem, F{∣a∣2}=F{a}⋆F{a}\mathcal{F}\{|a|^2\}=\mathcal{F}\{a\}\star\mathcal{F}\{a\}, with ⋆\star denoting correlation. Why: the transform of a squared modulus is the autocorrelation of the transform — the one algebraic identity that turns a real-space product into a frequency-space overlap integral.

Step 3. Substitute F{a}=F{F{P}}=P(−u,−v)\mathcal{F}\{a\}=\mathcal{F}\{\mathcal{F}\{P\}\}=P(-u,-v), so the OTF is the pupil autocorrelated with itself. Why: transforming twice returns the original function reflected through the origin, so correlating F{a}\mathcal{F}\{a\} with itself is precisely pupil-with-pupil.

Step 4. Normalize by the total energy ∬∣P∣2 du dv\iint|P|^2\,\mathrm{d}u\,\mathrm{d}v so H(0,0)=1H(0,0)=1, giving equation (4): full overlap at zero shift, shrinking overlap as the shift grows. Why: normalizing to unity at DC makes ∣H∣|H| a true contrast-transfer fraction between 0 and 1 — the quantity a bar-target measurement returns. *As a check: H(0,0)=1H(0,0)=1 (full overlap at zero shift), and because the support of an autocorrelation is twice the support of the original function, the incoherent cutoff falls at shift 2ρ02\rho_0 — exactly twice the pupil radius.*

Go deeper · Derivation B · 1 minDiffraction-limited MTF of a circular aperture
Derivation BDiffraction-limited MTF of a circular aperture

The goal here is to turn the autocorrelation theorem into the concrete curve plotted in the first figure — the MTF of a clear, aberration-free circular pupil under incoherent light. Because the pupil is real and uniform, its autocorrelation is pure geometry: the overlap area of two identical disks whose centres are separated by the shift s=λzifs=\lambda z_i f. We compute that lens-shaped overlap with circular-segment geometry, normalize by the pupil area, and change to the normalized frequency, arriving at the closed form every lens-design text quotes.

Result.

MTF(f~)=2π[arccos⁡f~−f~1−f~ 2],f~=ffc,  0≤f~≤1\mathrm{MTF}(\tilde f)=\dfrac{2}{\pi}\left[\arccos\tilde f-\tilde f\sqrt{1-\tilde f^{\,2}}\right],\qquad \tilde f=\dfrac{f}{f_c},\ \ 0\le\tilde f\le 1

Step 1. Two identical disks of radius ρ0\rho_0 with centres separated by ss overlap in a symmetric lens-shaped region; set the half-angle ϕ\phi by cos⁡ϕ=s/(2ρ0)\cos\phi=s/(2\rho_0). Why: parametrizing by ϕ\phi makes both the overlap area and the frequency come out as clean trigonometric functions.

Step 2. The overlap is twice a circular segment, A∩=2ρ02(ϕ−sin⁡ϕcos⁡ϕ)A_{\cap}=2\rho_0^2(\phi-\sin\phi\cos\phi). Why: each disk contributes one segment to the lens, and a segment of a disk of radius ρ0\rho_0 subtending half-angle ϕ\phi has area ρ02(ϕ−sin⁡ϕcos⁡ϕ)\rho_0^2(\phi-\sin\phi\cos\phi).

Step 3. Normalize by the pupil area, OTF=A∩/(πρ02)=2π(ϕ−sin⁡ϕcos⁡ϕ)\mathrm{OTF}=A_{\cap}/(\pi\rho_0^2)=\tfrac{2}{\pi}(\phi-\sin\phi\cos\phi). Why: the theorem requires dividing overlap by total area, and for a clear pupil there is no phase, so the OTF is real and positive and the PTF is zero.

Step 4. Substitute f~=f/fc=s/(2ρ0)=cos⁡ϕ\tilde f=f/f_c=s/(2\rho_0)=\cos\phi, so ϕ=arccos⁡f~\phi=\arccos\tilde f and sin⁡ϕcos⁡ϕ=f~1−f~ 2\sin\phi\cos\phi=\tilde f\sqrt{1-\tilde f^{\,2}}, giving the boxed form. Why: the cutoff is the shift s=2ρ0s=2\rho_0 where the disks separate, so measuring frequency in units of fcf_c is measuring the shift in units of the diameter.

Step 5. Endpoints: f~=0⇒MTF=1\tilde f=0\Rightarrow\mathrm{MTF}=1; f~=1⇒MTF=0\tilde f=1\Rightarrow\mathrm{MTF}=0; f~=12⇒arccos⁡12−123/2=1.0472−0.4330=0.6142\tilde f=\tfrac12\Rightarrow\arccos\tfrac12-\tfrac12\sqrt{3}/2=1.0472-0.4330=0.6142, so MTF=0.391\mathrm{MTF}=0.391. Why: a valid transfer function is unity at DC and zero at cutoff, and 0.391 is the standard mid-band checkpoint for validating MTF code and benches. *As a check: MTF(0)=1\mathrm{MTF}(0)=1, MTF(1)=0\mathrm{MTF}(1)=0, and MTF(0.5)=0.3910\mathrm{MTF}(0.5)=0.3910 — the standard mid-band value used to validate MTF code and test benches.*

Go deeper · Derivation C · 1 minThe Airy disk and the Rayleigh resolution criterion
Derivation CThe Airy disk and the Rayleigh resolution criterion

The aim of this derivation is to get the size of the diffraction blur in real image-plane units and connect it to the Rayleigh criterion. The route is Fraunhofer diffraction of a uniformly illuminated circular pupil, whose amplitude PSF is the jinc function built from the first-order Bessel function. The first zero of that Bessel function is a pure number, 3.8317, and dividing by π\pi produces the famous 1.22. Placing one Airy peak on a neighbour's first dark ring then defines the two-point resolution, tying the abstract cutoff back to a distance you can quote in nanometres.

Result.

rAiry=1.22 λF#=0.61 λNAr_{\mathrm{Airy}}=1.22\,\lambda F_\# = \dfrac{0.61\,\lambda}{\mathrm{NA}}

Step 1. The amplitude PSF is the jinc, a(r)∝2J1(kρ0r/zi)/(kρ0r/zi)a(r)\propto 2J_1(k\rho_0 r/z_i)/(k\rho_0 r/z_i). Why: the 2-D transform of a circular top-hat is a Hankel transform, whose value for a disk is exactly this jinc — the circular analogue of the slit's sinc.

Step 2. Square it, h(r)=∣a(r)∣2∝[2J1(x)/x]2h(r)=|a(r)|^2\propto[2J_1(x)/x]^2 with x=kρ0r/zix=k\rho_0 r/z_i; the first ring encloses about 83.8% of the energy. Why: the detector sees intensity, and the encircled-energy fraction fixes how concentrated the point image is.

Step 3. The first zero: J1(x)=0J_1(x)=0 at x=3.8317x=3.8317, so the first minimum sits at kρ0r/zi=3.8317k\rho_0 r/z_i=3.8317. Why: the edge of the Airy disk is where the diffracted amplitude first cancels — a fixed property of the Bessel function.

Step 4. Solve, r=3.8317 zi/(kρ0)=0.61 λzi/ρ0r=3.8317\,z_i/(k\rho_0)=0.61\,\lambda z_i/\rho_0, and use ρ0/zi≈NA\rho_0/z_i\approx\mathrm{NA} to get rAiry=0.61 λ/NAr_{\mathrm{Airy}}=0.61\,\lambda/\mathrm{NA}. Why: 3.8317/(2π)=0.6103.8317/(2\pi)=0.610 and 3.8317/π=1.2203.8317/\pi=1.220 are where the constants come from, and NA connects the radius directly to the cutoff of equation (5). *As a check: 3.8317/π=1.2197≈1.223.8317/\pi=1.2197\approx1.22, confirming the familiar constant; the encircled-energy fraction within the Airy disk is 0.838; and the Abbe limit λ/(2 NA)=1/fcinc\lambda/(2\,\mathrm{NA})=1/f_c^{\mathrm{inc}} is exactly the reciprocal of the incoherent cutoff.*

Limiting cases

CaseResultMeaning
Ideal, diffraction-limitedMTF(f~)=2π ⁣[arccos⁡f~−f~1−f~ 2]\mathrm{MTF}(\tilde f)=\tfrac{2}{\pi}\!\left[\arccos\tilde f-\tilde f\sqrt{1-\tilde f^{\,2}}\right]Clear circular aperture, incoherent. Monotonic, zero at fc=2 NA/λf_c=2\,\mathrm{NA}/\lambda — the best any lens of that NA can do.
Small aberrationσW≲λ/14\sigma_W\lesssim\lambda/14Marechal criterion: RMS wavefront error under λ/14\lambda/14 (≈λ/4\approx\lambda/4 peak-to-valley) keeps the Strehl ratio S≈e−(2πσW/λ)2S\approx e^{-(2\pi\sigma_W/\lambda)^2} above 0.8; the MTF stays near ideal.
Coherent illuminationH(f)=P(f), fccoh=NA/λH(f)=P(f),\ f_c^{\mathrm{coh}}=\mathrm{NA}/\lambdaTransfer equals the pupil — flat contrast, hard cutoff at half the incoherent value. Not an MTF.
Strong aberration or defocusOTF(f)<0\mathrm{OTF}(f)<0The OTF can dip below zero: spurious resolution and contrast reversal, black bars imaging white. The MTF hides the sign; the PTF records the π\pi jump.
Frequency at or above fcf_cMTF=0\mathrm{MTF}=0Detail finer than the cutoff radiates plane waves too steep to enter the pupil; that light is never collected, so it cannot be recovered by focusing or processing.

Worked examples

1 · Resolution of a microscope objective (the concrete anchor)

A dry objective has NA=0.95\mathrm{NA}=0.95 at λ=550\lambda=550 nm. Find its coherent and incoherent cutoffs, its Rayleigh and Abbe limits, and the contrast it transfers at the coherent cutoff.

Step 1. Coherent cutoff fccoh=NA/λ=0.95/0.550 μm=1.73f_c^{\mathrm{coh}}=\mathrm{NA}/\lambda=0.95/0.550\ \mu\mathrm{m}=1.73 cycles/μ\mum (≈1730\approx1730 lp/mm). Incoherent cutoff is twice this: fcinc=2 NA/λ=3.45f_c^{\mathrm{inc}}=2\,\mathrm{NA}/\lambda=3.45 cycles/μ\mum, about 3450 line-pairs/mm — the finest intensity grating it passes at all.

Step 2. Rayleigh limit dmin⁡Rayleigh=0.61λ/NA=0.61×550/0.95=353 nmd_{\min}^{\mathrm{Rayleigh}}=0.61\lambda/\mathrm{NA}=0.61\times550/0.95=\mathbf{353\ nm}. Abbe limit dmin⁡Abbe=λ/(2 NA)=550/1.90=289 nmd_{\min}^{\mathrm{Abbe}}=\lambda/(2\,\mathrm{NA})=550/1.90=\mathbf{289\ nm}, and as a check 1/fcinc=1/3.45 μm=2891/f_c^{\mathrm{inc}}=1/3.45\ \mu\mathrm{m}=289 nm — the Abbe limit is exactly the reciprocal of the incoherent cutoff.

Step 3. The coherent cutoff sits at f~=fccoh/fcinc=0.5\tilde f=f_c^{\mathrm{coh}}/f_c^{\mathrm{inc}}=0.5, so the incoherent MTF there is 2π[arccos⁡0.5−0.50.75]=0.391\tfrac{2}{\pi}[\arccos 0.5-0.5\sqrt{0.75}]=\mathbf{0.391} — a grating of that fineness images with only 39% of its contrast. To reach 200 nm one goes to oil immersion, NA≈1.4\mathrm{NA}\approx1.4, dropping the Abbe limit to 550/2.8=196550/2.8=196 nm. Cutoff 3450 lp/mm, Rayleigh 353 nm, Abbe 289 nm, MTF = 0.391 at the coherent cutoff.

2 · MTF of a camera lens at two test frequencies

A photographic lens works at f/8f/8 with λ=550\lambda=550 nm. Find the Airy-disk radius, the incoherent cutoff in lp/mm, and the MTF at 30 and 60 lp/mm.

Step 1. Airy radius rAiry=1.22 λF#=1.22×0.550 μm×8=5.37 μmr_{\mathrm{Airy}}=1.22\,\lambda F_\#=1.22\times0.550\ \mu\mathrm{m}\times8=\mathbf{5.37\ \mu m}. That exceeds a ∼4 μ\sim4\ \mum pixel, so at f/8f/8 the lens is diffraction-limited, not pixel-limited — stopping down further only softens it.

Step 2. Cutoff fcinc=1/(λF#)=1/(0.550×8) μm−1=227 lp/mmf_c^{\mathrm{inc}}=1/(\lambda F_\#)=1/(0.550\times8)\ \mu\mathrm{m}^{-1}=\mathbf{227\ lp/mm}. The test frequencies are f~=30/227=0.132\tilde f=30/227=0.132 and f~=60/227=0.264\tilde f=60/227=0.264.

Step 3. At 30 lp/mm: 2π[arccos⁡0.132−0.1321−0.1322]=2π[1.4384−0.1308]=0.832\tfrac{2}{\pi}[\arccos 0.132-0.132\sqrt{1-0.132^2}]=\tfrac{2}{\pi}[1.4384-0.1308]=\mathbf{0.832}. At 60 lp/mm: 2π[arccos⁡0.264−0.2641−0.2642]=2π[1.3035−0.2546]=0.668\tfrac{2}{\pi}[\arccos 0.264-0.264\sqrt{1-0.264^2}]=\tfrac{2}{\pi}[1.3035-0.2546]=\mathbf{0.668}. These are the ceilings a real f/8f/8 lens is graded against; aberrations only pull them lower. 83% contrast at 30 lp/mm, 67% at 60 lp/mm — the diffraction-limited best for f/8.

3 · Is a slightly defocused lens still diffraction-limited?

A lens carries residual defocus with peak-to-valley wavefront error λ/4\lambda/4. Estimate its RMS error and Strehl ratio, and decide.

Step 1. For defocus across a circular pupil the RMS is the peak-to-valley over 232\sqrt3: σW=(λ/4)/(23)=λ/13.9≈λ/14\sigma_W=(\lambda/4)/(2\sqrt3)=\lambda/13.9\approx\lambda/14 — which is why the "λ/4\lambda/4 PV" and "λ/14\lambda/14 RMS" rules are the same statement. The Strehl estimate is then S≈e−(2πσW/λ)2=e−(2π/14)2=e−0.2014=0.82S\approx e^{-(2\pi\sigma_W/\lambda)^2}=e^{-(2\pi/14)^2}=e^{-0.2014}=\mathbf{0.82}.

Step 2. The Marechal threshold is S>0.8S>0.8; since 0.82>0.800.82>0.80 the lens just qualifies as diffraction-limited — its MTF hugs the ideal curve, and the defocus costs a few percent of peak intensity without moving the cutoff. Any more defocus and the Strehl falls through 0.8 and the MTF visibly sags. RMS ≈λ/14\approx\lambda/14, Strehl ≈0.82\approx0.82 — yes, but barely.

Physics of the portrait

The transfer-function view is the modern language, but its physical heart — that resolution is set by how much of the diffracted spectrum an aperture can gather — was seen with startling clarity a century before Fourier optics had a name, by a man staring down a microscope for a lens company.

Origins · 1 minErnst Abbe and how this was discovered
EA
Ernst Abbe
1840–1905 · microscope image formation and the diffraction limit

Ernst Abbe joined Carl Zeiss's small Jena workshop in 1866 to replace trial-and-error lens grinding with theory, and in doing so founded the physics of image formation. His 1873 paper argued that a microscope does not simply magnify: it forms an image by diffraction. Illuminate a fine specimen and it acts as a grating, scattering light into discrete diffracted orders; the objective can reconstruct the specimen's structure only if it physically collects at least the zeroth and first orders. Detail finer than d=λ/(2 NA)d=\lambda/(2\,\mathrm{NA}) scatters its first order outside the aperture, and no polishing, magnification or eyepiece recovers it — the information never entered the lens. This is precisely the cutoff 1/fcinc1/f_c^{\mathrm{inc}} of this article, reached by counting diffraction orders instead of autocorrelating pupils. To exploit it Abbe defined and named the numerical aperture NA=nsin⁡θ\mathrm{NA}=n\sin\theta, championed oil immersion to push NA above unity, and with the glassmaker Otto Schott built the first apochromatic objectives from theory. His diffraction theory, resolution formula and numerical aperture are the direct ancestors of the OTF; the Rayleigh, Sparrow, Duffieux and Hopkins refinements only sharpen the boundary Abbe first drew.

History
1835
— Airy computes the diffraction pattern of a circular aperture, the Airy disk
1873
— Abbe's diffraction theory of image formation gives the limit d = λ/(2·NA)
1879
— Rayleigh publishes the two-point resolution criterion
1916
— Sparrow proposes the vanishing-dip resolution criterion
1946
— Duffieux introduces Fourier methods and the transfer function into optics
1955
— Hopkins formalizes the optical transfer function for partially coherent imaging

Going further

Texts. Goodman, Introduction to Fourier Optics, 3rd ed., Ch. 6 (coherent and incoherent imaging, the OTF) · Hecht, Optics, 5th ed., Ch. 13 (Fourier optics, OTF and MTF) and Ch. 10 (Fraunhofer diffraction, the Airy pattern) · Born and Wolf, Principles of Optics, 7th ed., Ch. 8-9 (diffraction and aberrations) · Saleh and Teich, Fundamentals of Photonics, 3rd ed., Ch. 4 · Demtroeder, Laser Spectroscopy, Vol. 1 (spectrographs and imaging resolution). Key numerical anchors for this article: the circular-aperture MTF satisfies MTF(0)=1\mathrm{MTF}(0)=1, MTF(1)=0\mathrm{MTF}(1)=0, MTF(0.5)=0.391\mathrm{MTF}(0.5)=0.391; the Rayleigh radius follows from the first zero of J1J_1 at 3.8317, giving 3.8317/π=1.2203.8317/\pi=1.220 and r=0.61 λ/NAr=0.61\,\lambda/\mathrm{NA}; the Airy disk encircles 0.838 of the total energy; the cutoffs are fcinc=2 NA/λ=1/(λF#)f_c^{\mathrm{inc}}=2\,\mathrm{NA}/\lambda=1/(\lambda F_\#) and fccoh=NA/λf_c^{\mathrm{coh}}=\mathrm{NA}/\lambda; the Strehl estimate gives S≈e−(2πσW/λ)2=0.82S\approx e^{-(2\pi\sigma_W/\lambda)^2}=0.82 at σW=λ/14\sigma_W=\lambda/14; and the worked MTF values are 0.832 at f~=0.132\tilde f=0.132 and 0.668 at f~=0.264\tilde f=0.264.

Prerequisites. Numerical Aperture and Acceptance Angle · Fourier Transforms and Convolution · Fraunhofer Diffraction. Leads to. Integrated Photonic Waveguides and Circuits · Microring Resonators and Optical Cavities on Chip · Total Internal Reflection and Optical Guiding.

Ask the tutor: Tell me your lens or objective's NA (or f-number) and wavelength, and I will walk you through its cutoff, Airy disk, and full MTF curve step by step.

What is the difference between the PSF, the OTF and the MTF?

They are the same information in two domains. The point-spread function is the image of a single ideal point — the blur kernel in real space, so an image is the object convolved with it. The optical transfer function is the Fourier transform of the PSF: the same instrument in the spatial-frequency domain, where convolution becomes multiplication. The OTF is complex; its modulus is the MTF, the fraction of contrast transferred at each frequency, and its phase is the PTF, telling you whether the pattern shifts or inverts. A compact PSF means a broad OTF means high MTF at fine detail — a sharp lens.

Why can't I recover detail beyond the cutoff by sharpening or deconvolution?

Because the light was never collected. Detail finer than 1/fc1/f_c radiates plane waves tilted more steeply than the aperture accepts, so those frequencies arrive with exactly zero amplitude — the OTF is identically zero there. Sharpening and deconvolution only boost frequencies the instrument passed but attenuated; where the OTF is zero there is no signal to boost, only noise, and dividing by zero amplifies that noise without bound. The diffraction limit is a wall of missing information, not merely low contrast. Real super-resolution beats it only by adding information the snapshot lacks — nonlinearity, structured illumination, or prior knowledge of the object.

Is the MTF a property of the lens alone?

No — it is a property of the whole imaging chain, and that is its convenience. Because each shift-invariant stage multiplies in the frequency domain, the system MTF is the product of the individual MTFs: the lens (diffraction plus aberrations), the detector's finite pixel aperture (a sinc), any anti-alias filter, even motion blur or turbulence. A superb lens behind a coarse sensor gives a mediocre system, and the cascade shows exactly where contrast is lost. This is why engineers budget MTF stage by stage, and why a lens MTF chart never tells you the final image sharpness on its own.

How do the Rayleigh, Abbe and Sparrow criteria differ?

They are three conventions for "just resolved," differing only in a constant, all proportional to λ/NA\lambda/\mathrm{NA}. Rayleigh (0.61 λ/NA0.61\,\lambda/\mathrm{NA}) places one Airy peak on its neighbour's first dark ring, leaving a roughly 26% dip — a conservative rule. Abbe (λ/2 NA\lambda/2\,\mathrm{NA}, finer) is the grating-period limit, the reciprocal of the incoherent cutoff and the most fundamental. Sparrow (0.47 λ/NA0.47\,\lambda/\mathrm{NA}, finer still) marks where the central dip just vanishes. None is more correct; they pick different points on the same continuous MTF, which is the complete answer they crudely summarize.

What does it mean when the OTF goes negative?

It means contrast reversal, also called spurious resolution. A well-corrected clear aperture has a strictly positive MTF, but a defocused or strongly aberrated system can drive the OTF below zero beyond its first null. At such a frequency a black-and-white bar pattern images with the bars swapped — dark comes out bright — because the phase transfer function has jumped by π\pi. The pattern is still "resolved" in that structure is visible, but it is a faithful liar: the fine detail is present yet inverted. This is why the modulus alone is not the whole story; you need the PTF to know whether the instrument is telling the truth at a given frequency.

Why isn't coherent imaging simply better for having twice the contrast?

Because its cutoff is half as large and its behaviour is fundamentally different. A coherent system is linear in complex field, so its transfer function is the pupil itself — flat unit contrast, but stopping hard at NA/λ\mathrm{NA}/\lambda, half the incoherent cutoff. It also produces speckle, edge ringing, and a two-point response that depends on the relative phase of the sources, none of which an MTF captures. So flat contrast comes with a lower resolution ceiling and coherent artefacts, while the incoherent system trades flat contrast for graded contrast reaching twice as far. They are different linear systems — one in field, one in intensity — to be compared on their own terms, not ranked.

Before this

Where this leads