The Neyman-Pearson Lemma

Suppose a quality-control engineer must decide, from a single sample, whether a batch of components meets specification or fails it. She fixes a tolerance for false alarms — say, at most a 5% chance of rejecting a good batch. Then she asks: among all decision rules with that false-alarm rate, which one catches the most true failures? This is the essence of hypothesis testing reduced to its purest form. The Neyman–Pearson Lemma answers it completely when the two competing hypotheses are both simple (each fully specifying a probability distribution). The lemma matters because it converts an apparently open-ended optimization problem into an explicit recipe. The problem is to search over all possible tests, an infinite-dimensional space of decision rules. The recipe is simple: compute a likelihood ratio, compare it to a threshold. This single computation, repeated across thousands of applications, is the ancestor of nearly every modern testing procedure. T-tests, chi-squared tests, and the general Likelihood Ratio Tests used throughout statistics all trace their optimality intuition back to this result. The big picture is that "best test" needs a precise meaning before it can be found. Neyman and Pearson supplied it: fix the probability of a false positive (Type I error) at a level α\alpha. Then maximize the probability of correctly detecting the alternative (power). Among all tests satisfying the error constraint, the lemma identifies the one that maximizes power. This optimal test is built from the likelihood ratio. The lemma also proves no other test can do better. It is an extremal result of the highest classical kind. It gives an explicit optimizer, together with a proof that nothing else in the entire feasible set beats it. This article states the Hypothesis Testing Framework precisely, and defines the likelihood ratio test. It proves the Neyman–Pearson Lemma completely (existence, optimality, and the converse). It works through two asymmetric numerical examples. Finally, it shows exactly which hypothesis breaks the result when discarded.

Key takeaways
  • The Neyman–Pearson Lemma finds the most powerful test for two simple hypotheses at a fixed significance level α\alpha.
  • The optimal test rejects H0H_0 when the likelihood ratio Λ(x)=f1(x)/f0(x)\Lambda(x) = f_1(x)/f_0(x) exceeds a threshold kk.
  • The threshold kk and any randomization are chosen so the test has exact size α\alpha.
  • The proof is a direct comparison argument: integrate the sign of (ϕ−ϕ∗)(f1−kf0)(\phi - \phi^*)(f_1 - kf_0) over the sample space.
  • The lemma also proves a converse: any test of this likelihood-ratio form is automatically most powerful for its own size.
  • Key idea: among all size-α\alpha tests, thresholding the likelihood ratio is the unique (up to ties) way to maximize power.

Where this lives today

The Neyman–Pearson Lemma is the theoretical seed of modern hypothesis testing and appears far beyond introductory statistics courses. In signal detection theory and radar/sonar engineering, the lemma is the mathematical basis of the optimal detector: distinguishing "signal present" from "noise only" from a received waveform, where the likelihood-ratio threshold directly yields the receiver operating characteristic (ROC) curve used to trade false alarms against missed detections. In digital communications, matched-filter receivers implementing maximum-likelihood detection of transmitted bits are literally likelihood-ratio tests between simple hypotheses (which symbol was sent). In cryptography and information-theoretic security, likelihood-ratio arguments underlie bounds on an adversary's ability to distinguish two distributions (e.g. a real cipher output from random noise), connecting directly to statistical distance and hypothesis-testing divergences. In machine learning, the Neyman–Pearson framework has been revived explicitly as "NP classification," where a classifier is trained to minimize one type of error subject to a hard constraint on the other — directly generalizing the lemma's asymmetric error treatment to high-dimensional, non-parametric settings. In quality control and clinical trials, the lemma justifies why a pre-specified test statistic and rejection region, chosen before seeing data, is provably the most powerful way to detect a specified effect size at a fixed false-positive rate. Finally, within pure statistics, the lemma is the base case from which the broader theory of Likelihood Ratio Tests and uniformly most powerful tests under monotone likelihood ratio conditions is developed.

Ask the tutor: How does the Neyman–Pearson Lemma generalize when testing one-sided composite hypotheses using the monotone likelihood ratio property?

DefinitionSimple hypothesis testing problem and most powerful test

Let XX be an observation (possibly a vector) taking values in a sample space X\mathcal{X}, and suppose its distribution is either P0P_0 or P1P_1, two probability measures on X\mathcal{X} that are mutually absolutely continuous with respect to a common dominating measure μ\mu (e.g. Lebesgue measure or counting measure), with densities f0=dP0/dμf_0 = dP_0/d\mu and f1=dP1/dμf_1 = dP_1/d\mu. Consider the simple versus simple hypotheses

H0:X∼P0vs.H1:X∼P1.H_0 : X \sim P_0 \qquad \text{vs.} \qquad H_1 : X \sim P_1.

A test is a measurable function ϕ:X→[0,1]\phi : \mathcal{X} \to [0,1], interpreted as follows: upon observing X=xX = x, the statistician rejects H0H_0 with probability ϕ(x)\phi(x) (independently of everything else, via auxiliary randomization if ϕ(x)∉{0,1}\phi(x) \notin \{0,1\}). The quantities

αϕ=E0[ϕ(X)](size / Type I error probability),\alpha_\phi = E_0[\phi(X)] \qquad (\text{size / Type I error probability}),
βϕ=E1[1−ϕ(X)](Type II error probability),πϕ=E1[ϕ(X)]=1−βϕ(power)\beta_\phi = E_1[1 - \phi(X)] \qquad (\text{Type II error probability}), \qquad \pi_\phi = E_1[\phi(X)] = 1-\beta_\phi \quad (\text{power})

describe its performance. Fix a level α∈[0,1]\alpha \in [0,1]. A test ϕ∗\phi^* is a most powerful (MP) test of size α\alpha if: 1. E0[ϕ∗(X)]=αE_0[\phi^*(X)] = \alpha (or ≤α\le \alpha, with the convention below), and 2. for every test ϕ\phi with E0[ϕ(X)]≤αE_0[\phi(X)] \le \alpha, we have E1[ϕ∗(X)]≥E1[ϕ(X)]E_1[\phi^*(X)] \ge E_1[\phi(X)]. That is, ϕ∗\phi^* maximizes power among all tests whose Type I error probability does not exceed α\alpha.

Intuition — prose building the mental picture

Think of every point xx in the sample space as casting a vote for H0H_0 or H1H_1, and the strength of that vote is the likelihood ratio Λ(x)=f1(x)/f0(x)\Lambda(x) = f_1(x)/f_0(x): how much more plausible is this observation under H1H_1 than under H0H_0? A rational tester should reject H0H_0 precisely on the observations that most strongly favor H1H_1 — the points with the largest likelihood ratios — because those are exactly the points that buy the most power per unit of Type I error spent. Picture the sample space partitioned and re-sorted by the value of Λ(x)\Lambda(x), largest to smallest. As you sweep a threshold kk down from +∞+\infty, you accumulate rejection region {x:Λ(x)>k}\{x : \Lambda(x) > k\}, and its probability under P0P_0 (the false-alarm rate) grows continuously from 00 towards 11 while its probability under P1P_1 (the power) also grows. The Neyman–Pearson Lemma says this particular sweep is efficient: for any fixed budget α\alpha of false-alarm probability, no other way of choosing a rejection region of that same P0P_0-probability can capture more P1P_1-probability. Buying "expensive" points (low likelihood ratio) instead of "cheap" ones (high likelihood ratio) can only waste your error budget.

Figure
Sorting outcomes by likelihood ratio: the optimal rejection region takes the largest values of Λ first
Sample points sorted by likelihood ratio Λ(x) = f₁(x)/f₀(x) Λ decreasing → large Λ threshold k rejection region {Λ > k} : size α mass under P₁ (power) mass under P₀ (size)
TheoremNeyman–Pearson Lemma

Let XX have density f0f_0 under H0H_0 and f1f_1 under H1H_1 with respect to a common dominating measure μ\mu on X\mathcal{X}. Fix α∈(0,1)\alpha \in (0,1).

(a) Existence. There exist a constant k∈[0,∞]k \in [0,\infty] and a number γ∈[0,1]\gamma \in [0,1] such that the test

ϕ∗(x)={1if f1(x)>kf0(x),γif f1(x)=kf0(x),0if f1(x)<kf0(x),\phi^*(x) = \begin{cases} 1 & \text{if } f_1(x) > k f_0(x), \\ \gamma & \text{if } f_1(x) = k f_0(x), \\ 0 & \text{if } f_1(x) < k f_0(x), \end{cases}

satisfies E0[ϕ∗(X)]=αE_0[\phi^*(X)] = \alpha exactly.

(b) Optimality (sufficiency). Any such test ϕ∗\phi^* is most powerful of size α\alpha: for every test ϕ\phi with E0[ϕ(X)]≤αE_0[\phi(X)] \le \alpha,

E1[ϕ(X)]≤E1[ϕ∗(X)].E_1[\phi(X)] \le E_1[\phi^*(X)].

(c) Necessity (converse). If ϕ∗\phi^* is most powerful of size α\alpha (for some k<∞k<\infty) then, except possibly on a set NN with P0(N)=P1(N)=0P_0(N) = P_1(N) = 0, ϕ∗\phi^* has the form above for some k≥0k \ge 0; moreover if ϕ\phi is any most powerful size-α\alpha test then ϕ=ϕ∗\phi = \phi^* a.e.-P0P_0 and a.e.-P1P_1 on {f1≠kf0}\{f_1 \ne k f_0\}, unless there is a test of size strictly less than α\alpha with power 11.

Proof

We prove each part in turn. Throughout, write E0,E1E_0, E_1 for expectation under P0,P1P_0, P_1, and ∫\int for ∫X(⋅) dμ\int_{\mathcal X} (\cdot)\, d\mu.

(a) Existence of kk and γ\gamma. Define G(t)=P0(f1(X)>tf0(X))G(t) = P_0(f_1(X) > t f_0(X)) for t∈[0,∞)t \in [0,\infty), the survival function of the likelihood ratio under P0P_0 (with the convention f1(x)/f0(x)=∞f_1(x)/f_0(x) = \infty interpreted appropriately when f0(x)=0f_0(x)=0, using Λ(x)>t\Lambda(x)>t meaning f1(x)>tf0(x)f_1(x) > t f_0(x), which is well defined even when f0(x)=0f_0(x) = 0). This GG is monotone non-increasing and right-continuous (a standard property of survival functions of a real-valued random variable Λ(X)\Lambda(X) under P0P_0), with G(0−)G(0^-) effectively 11 and G(t)→0G(t) \to 0 as t→∞t \to \infty since P0P_0 is a probability measure and {f1>tf0}\{f_1 > tf_0\} shrinks to a P0P_0-null set as t→∞t\to\infty (because ∫f1 dμ=1<∞\int f_1 \,d\mu =1<\infty forces f1(x)>tf0(x)f_1(x) > t f_0(x) to fail μ\mu-a.e. as t→∞t\to\infty on the set f0>0f_0>0, and the set f0=0f_0 = 0 has P0P_0-measure zero by definition of density). Set

k=inf⁡{t≥0:G(t)≤α}.k = \inf\{ t \ge 0 : G(t) \le \alpha \}.

This infimum is finite and well-defined since GG is non-increasing, right-continuous, decreasing to 00; standard properties of generalized inverses of monotone right-continuous functions give

G(k)≤α≤G(k−):=lim⁡t↑kG(t).G(k) \le \alpha \le G(k^-) := \lim_{t \uparrow k} G(t).

(If k=0k=0, read G(0−)G(0^-) as 11.) The jump G(k−)−G(k)=P0(f1(X)=kf0(X))G(k^-) - G(k) = P_0(f_1(X) = k f_0(X)), since

G(k−)−G(k)=P0(f1>kf0)−P0(f1>kf0) in the limit=P0(f1(X)=kf0(X)),G(k^-) - G(k) = P_0(f_1 > kf_0) - P_0(f_1>kf_0) \text{ in the limit} = P_0\big(f_1(X) = k f_0(X)\big),

by definition of kk as the point where the survival function of Λ(X)\Lambda(X) under P0P_0 jumps past level α\alpha (this is exactly the standard construction of a randomized quantile). If P0(f1=kf0)=0P_0(f_1 = kf_0) = 0, we may take γ\gamma arbitrary (say γ=0\gamma = 0) and E0[ϕ∗(X)]=G(k)=αE_0[\phi^*(X)] = G(k) = \alpha automatically forces G(k)=αG(k)=\alpha in this case. Otherwise, define

γ=α−G(k)P0(f1(X)=kf0(X))∈[0,1],\gamma = \frac{\alpha - G(k)}{P_0(f_1(X) = kf_0(X))} \in [0,1],

which lies in [0,1][0,1] precisely because G(k)≤α≤G(k−)=G(k)+P0(f1=kf0)G(k) \le \alpha \le G(k^-) = G(k) + P_0(f_1=kf_0). Then

E0[ϕ∗(X)]=P0(f1>kf0)+γ P0(f1=kf0)=G(k)+(α−G(k))=α,E_0[\phi^*(X)] = P_0(f_1 > kf_0) + \gamma\, P_0(f_1 = kf_0) = G(k) + (\alpha - G(k)) = \alpha,

establishing (a).

(b) Optimality. Let ϕ\phi be any test with E0[ϕ(X)]≤α=E0[ϕ∗(X)]E_0[\phi(X)] \le \alpha = E_0[\phi^*(X)]. We must show E1[ϕ∗(X)]≥E1[ϕ(X)]E_1[\phi^*(X)] \ge E_1[\phi(X)], equivalently

E1[ϕ∗(X)−ϕ(X)]≥0.E_1[\phi^*(X) - \phi(X)] \ge 0.

The key pointwise inequality is: for every x∈Xx \in \mathcal{X},

(ϕ∗(x)−ϕ(x))(f1(x)−kf0(x))≥0.(∗)\big(\phi^*(x) - \phi(x)\big)\big(f_1(x) - k f_0(x)\big) \ge 0. \tag{$\ast$}

To see this, consider the three cases in the definition of ϕ∗\phi^*:

  • If f1(x)>kf0(x)f_1(x) > k f_0(x), then ϕ∗(x)=1≥ϕ(x)\phi^*(x) = 1 \ge \phi(x), so ϕ∗(x)−ϕ(x)≥0\phi^*(x) - \phi(x) \ge 0, and f1(x)−kf0(x)>0f_1(x) - kf_0(x) > 0; the product is ≥0\ge 0.
  • If f1(x)<kf0(x)f_1(x) < k f_0(x), then ϕ∗(x)=0≤ϕ(x)\phi^*(x) = 0 \le \phi(x), so ϕ∗(x)−ϕ(x)≤0\phi^*(x) - \phi(x) \le 0, and f1(x)−kf0(x)<0f_1(x) - kf_0(x) < 0; the product is again ≥0\ge 0 (negative times negative).
  • If f1(x)=kf0(x)f_1(x) = k f_0(x), the second factor is 00, so the product is 0≥00 \ge 0.

In every case (∗\ast) holds. Since (∗)(\ast) holds for every xx, integrating against μ\mu preserves the inequality:

∫(ϕ∗(x)−ϕ(x))(f1(x)−kf0(x)) dμ(x)≥0.\int \big(\phi^*(x) - \phi(x)\big)\big(f_1(x) - k f_0(x)\big)\, d\mu(x) \ge 0.

Expand the integrand and use ∫ϕfi dμ=Ei[ϕ(X)]\int \phi f_i \, d\mu = E_i[\phi(X)] (valid since fif_i is the density of PiP_i w.r.t. μ\mu):

E1[ϕ∗(X)]−E1[ϕ(X)]−k(E0[ϕ∗(X)]−E0[ϕ(X)])≥0.E_1[\phi^*(X)] - E_1[\phi(X)] - k\Big(E_0[\phi^*(X)] - E_0[\phi(X)]\Big) \ge 0.

Now E0[ϕ∗(X)]=α≥E0[ϕ(X)]E_0[\phi^*(X)] = \alpha \ge E_0[\phi(X)] by hypothesis, so E0[ϕ∗(X)]−E0[ϕ(X)]≥0E_0[\phi^*(X)] - E_0[\phi(X)] \ge 0. Since k≥0k \ge 0, we get

−k(E0[ϕ∗(X)]−E0[ϕ(X)])≤0.-k\Big(E_0[\phi^*(X)] - E_0[\phi(X)]\Big) \le 0.

Combining,

E1[ϕ∗(X)]−E1[ϕ(X)]≥k(E0[ϕ∗(X)]−E0[ϕ(X)])≥0,E_1[\phi^*(X)] - E_1[\phi(X)] \ge k\Big(E_0[\phi^*(X)] - E_0[\phi(X)]\Big) \ge 0,

which is exactly E1[ϕ∗(X)]≥E1[ϕ(X)]E_1[\phi^*(X)] \ge E_1[\phi(X)]. This proves (b): ϕ∗\phi^* is most powerful of size α\alpha. (Note this argument required only k≥0k\ge 0 and the pointwise inequality (∗)(\ast); it did not use any specific numeric value beyond nonnegativity of kk, so it is fully general.)

(c) Necessity / converse. Suppose ϕ∗\phi^* as constructed in (a) is most powerful of size α\alpha (automatic from (b)), and let ϕ1\phi_1 be any most powerful size-α\alpha test. Since both are most powerful of size α\alpha, they achieve the same maximal power:

E1[ϕ1(X)]=E1[ϕ∗(X)]=:π∗.E_1[\phi_1(X)] = E_1[\phi^*(X)] =: \pi^*.

We now show ϕ1\phi_1 must agree with ϕ∗\phi^* except on a (P0,P1)(P_0,P_1)-null set, under the non-degenerate assumption that no test of size strictly less than α\alpha attains power 11 (this rules out a trivial case where the constraint E0[ϕ]≤αE_0[\phi]\le \alpha is not binding at all, e.g. P0=P1P_0=P_1 mutually singular support issues; this is a standard regularity exclusion). Re-run the integration in part (b) with ϕ=ϕ1\phi = \phi_1. Because E1[ϕ1]=E1[ϕ∗]E_1[\phi_1] = E_1[\phi^*], the final chain of inequalities in part (b) — which was

0  ≤  ∫(ϕ∗−ϕ1)(f1−kf0) dμ  =  (E1[ϕ∗]−E1[ϕ1])−k(E0[ϕ∗]−E0[ϕ1])0 \;\le\; \int (\phi^*-\phi_1)(f_1 - kf_0)\,d\mu \;=\; \big(E_1[\phi^*]-E_1[\phi_1]\big) - k\big(E_0[\phi^*]-E_0[\phi_1]\big)

— must in fact equal 00 on both ends: the left side is 00 (since E1[ϕ∗]−E1[ϕ1]=0E_1[\phi^*]-E_1[\phi_1]=0) and it also equals −k(E0[ϕ∗]−E0[ϕ1])-k(E_0[\phi^*]-E_0[\phi_1]) where E0[ϕ1]≤α=E0[ϕ∗]E_0[\phi_1]\le \alpha = E_0[\phi^*]; since k≥0k \ge 0 this term is ≤0\le 0 and we already showed the integral is ≥0\ge 0, forcing

∫(ϕ∗−ϕ1)(f1−kf0) dμ=0.\int (\phi^*-\phi_1)(f_1-kf_0)\,d\mu = 0.

But the integrand (ϕ∗(x)−ϕ1(x))(f1(x)−kf0(x))(\phi^*(x)-\phi_1(x))(f_1(x)-kf_0(x)) is pointwise non-negative by the identical case analysis as in (∗\ast) applied with ϕ1\phi_1 in place of ϕ\phi (the argument used only that ϕ1(x)∈[0,1]\phi_1(x) \in [0,1], which holds for any test). A non-negative function with zero integral against μ\mu must vanish μ\mu-a.e., hence

(ϕ∗(x)−ϕ1(x))(f1(x)−kf0(x))=0for μ-a.e. x.\big(\phi^*(x) - \phi_1(x)\big)\big(f_1(x) - kf_0(x)\big) = 0 \quad \text{for } \mu\text{-a.e. } x.

On the set {f1(x)≠kf0(x)}\{f_1(x) \ne k f_0(x)\} this forces ϕ1(x)=ϕ∗(x)\phi_1(x) = \phi^*(x) for μ\mu-a.e. such xx, and since f0,f1f_0, f_1 are densities of P0,P1P_0, P_1 this equality holds P0P_0-a.e. and P1P_1-a.e. on that set as well (a μ\mu-null set is automatically P0P_0-null and P1P_1-null because P0,P1≪μP_0,P_1 \ll \mu). This proves ϕ1\phi_1 agrees with ϕ∗\phi^* off the boundary set {f1=kf0}\{f_1 = kf_0\}, which is the asserted essential uniqueness, completing (c). ■\blacksquare

∎

Proof technique used: a direct pointwise sign/comparison argument (sometimes called the "Neyman–Pearson trick") integrated over the sample space, combined with a generalized-inverse construction of the randomization threshold from the survival function of the likelihood ratio. No result depending on Neyman–Pearson was used; the construction of k,γk,\gamma uses only monotonicity of measures, and the optimality argument uses only linearity of expectation and the sign analysis, so the proof is self-contained and non-circular.

Worked examples

Example 1 — Exponential distributions, asymmetric rates. Let X1,…,XnX_1,\dots,X_n be i.i.d. with density f(x;θ)=θe−θxf(x;\theta) = \theta e^{-\theta x} for x>0x>0. Test

H0:θ=θ0=2vs.H1:θ=θ1=5,H_0: \theta = \theta_0 = 2 \qquad \text{vs.} \qquad H_1 : \theta = \theta_1 = 5,

at level α=0.05\alpha = 0.05, with n=10n = 10 observations. The joint densities are fi(x)=θinexp⁡(−θi∑xj)f_i(\mathbf{x}) = \theta_i^n \exp(-\theta_i \sum x_j) for i=0,1i=0,1. The likelihood ratio is

Λ(x)=f1(x)f0(x)=(θ1θ0)nexp⁡(−(θ1−θ0)∑xj)=2.510 e−3T,T:=∑j=110Xj.\Lambda(\mathbf x) = \frac{f_1(\mathbf x)}{f_0(\mathbf x)} = \left(\frac{\theta_1}{\theta_0}\right)^n \exp\big(-(\theta_1-\theta_0)\textstyle\sum x_j\big) = 2.5^{10}\, e^{-3 T}, \qquad T := \sum_{j=1}^{10} X_j.

Since θ1>θ0\theta_1 > \theta_0, the exponent coefficient −(θ1−θ0)=−3<0-(\theta_1-\theta_0) = -3 <0, so Λ(x)\Lambda(\mathbf x) is a strictly decreasing function of TT. Hence Λ(x)>k  ⟺  T<c\Lambda(\mathbf x) > k \iff T < c for some constant cc determined by kk (a monotone decreasing transform reverses the inequality). By the Neyman–Pearson Lemma, the most powerful test rejects H0H_0 for small TT:

ϕ∗(x)=1{T<c}.\phi^*(\mathbf x) = \mathbb 1\{T < c\}.

No randomization is needed here because TT has a continuous distribution under P0P_0, so we can choose cc to make the size exactly 0.050.05 without a boundary atom. Under H0H_0 (θ0=2\theta_0=2), T=∑Xj∼Gamma(n=10,rate=2)T = \sum X_j \sim \mathrm{Gamma}(n=10, \text{rate}=2), so 2θ0T=4T∼χ2022\theta_0 T = 4T \sim \chi^2_{20} (using the standard fact that a sum of nn i.i.d. Exp(θ)\mathrm{Exp}(\theta) variables scaled by 2θ2\theta is χ2n2\chi^2_{2n}). We need

P0(T<c)=0.05⟺P(χ202<4c)=0.05.P_0(T < c) = 0.05 \quad\Longleftrightarrow\quad P\big(\chi^2_{20} < 4c\big) = 0.05.

The 55th percentile of χ202\chi^2_{20} is 10.85110.851 (a standard tabulated value), so 4c=10.8514c = 10.851, giving

c=2.71275.c = 2.71275.

The most powerful size-0.050.05 test rejects H0:θ=2H_0:\theta=2 in favor of H1:θ=5H_1:\theta=5 whenever ∑j=110Xj<2.71275\sum_{j=1}^{10} X_j < 2.71275. Its power is P1(T<c)P_1(T < c) where under H1H_1, 2θ1T=10T∼χ2022\theta_1 T = 10T \sim \chi^2_{20}, so

power=P(χ202<10×2.71275)=P(χ202<27.1275).\text{power} = P(\chi^2_{20} < 10 \times 2.71275) = P(\chi^2_{20} < 27.1275).

Using the χ202\chi^2_{20} distribution, P(χ202<27.1275)≈0.869P(\chi^2_{20} < 27.1275) \approx 0.869 (the χ202\chi^2_{20} distribution has 8080th percentile near 28.4128.41 and 7575th near 25.0425.04, so 27.1327.13 lands close to the 8787th percentile by interpolation). Thus this test correctly detects the higher rate about 87%87\% of the time while falsely rejecting a true H0H_0 only 5%5\% of the time — and by the lemma, no other level-0.050.05 test based on this sample can do better.

Example 2 — Discrete asymmetric Bernoulli-type problem with randomization. Let X∼Binomial(n=8,p)X \sim \mathrm{Binomial}(n=8, p), and test

H0:p=0.3vs.H1:p=0.6,H_0: p = 0.3 \qquad \text{vs.} \qquad H_1 : p = 0.6,

at level α=0.10\alpha = 0.10. The likelihood ratio for observed count x∈{0,…,8}x \in \{0,\dots,8\} is

Λ(x)=(8x)(0.6)x(0.4)8−x(8x)(0.3)x(0.7)8−x=(0.60.3)x(0.40.7)8−x=2x(47)8−x,\Lambda(x) = \frac{\binom{8}{x}(0.6)^x(0.4)^{8-x}}{\binom{8}{x}(0.3)^x(0.7)^{8-x}} = \left(\frac{0.6}{0.3}\right)^x \left(\frac{0.4}{0.7}\right)^{8-x} = 2^x \left(\frac{4}{7}\right)^{8-x},

which is increasing in xx (each unit increase in xx multiplies Λ\Lambda by 2/(4/7)=3.5>12/(4/7) = 3.5 > 1). So the most powerful test rejects for large XX: ϕ∗(x)=1\phi^*(x) = 1 if x>cx > c, possibly randomized at x=cx=c. Compute the null (p=0.3p=0.3) binomial pmf for n=8n=8:

P0(X=k)=(8k)(0.3)k(0.7)8−k.P_0(X=k) = \binom{8}{k}(0.3)^k(0.7)^{8-k}.

Numerically: P0(X=8)=0.38=0.0000656P_0(X=8)=0.3^8 = 0.0000656, P0(X=7)=8(0.3)7(0.7)=0.0012247P_0(X=7)=8(0.3)^7(0.7)=0.0012247, P0(X=6)=28(0.3)6(0.7)2=0.0100033P_0(X=6)=28(0.3)^6(0.7)^2 = 0.0100033, P0(X=5)=56(0.3)5(0.7)3=0.0466714P_0(X=5) = 56(0.3)^5(0.7)^3 = 0.0466714. Cumulative upper tail probabilities under H0H_0:

P0(X≥8)=0.0000656,P0(X≥7)=0.0012903,P0(X≥6)=0.0112936,P0(X≥5)=0.0579650.P_0(X\ge 8) = 0.0000656,\quad P_0(X\ge7)=0.0012903,\quad P_0(X\ge6)=0.0112936,\quad P_0(X\ge5)=0.0579650.

We want size exactly α=0.10\alpha=0.10. Since P0(X≥5)=0.05797<0.10<P0(X≥4)P_0(X\ge5)=0.05797 < 0.10 < P_0(X\ge4), the threshold sits between 44 and 55: reject outright for X≥5X\ge5, and randomize at X=4X=4. Compute P0(X=4)=70(0.3)4(0.7)4=70×0.0081×0.2401=0.1361P_0(X=4) = 70(0.3)^4(0.7)^4 = 70 \times 0.0081 \times 0.2401 = 0.1361 (rounded). Then

P0(X≥4)=P0(X≥5)+P0(X=4)=0.05797+0.1361=0.19407.P_0(X\ge 4) = P_0(X\ge5) + P_0(X=4) = 0.05797 + 0.1361 = 0.19407.

We need

P0(X≥5)+γ P0(X=4)=0.10⟹γ=0.10−0.057970.1361=0.042030.1361≈0.3088.P_0(X\ge5) + \gamma\, P_0(X=4) = 0.10 \quad\Longrightarrow\quad \gamma = \frac{0.10 - 0.05797}{0.1361} = \frac{0.04203}{0.1361} \approx 0.3088.

So the most powerful size-0.100.10 test is

ϕ∗(x)={1x≥5,0.3088x=4,0x≤3,\phi^*(x) = \begin{cases} 1 & x \ge 5,\\ 0.3088 & x = 4,\\ 0 & x \le 3,\end{cases}

meaning: if X=4X=4 is observed, flip a biased coin (reject with probability ≈0.309\approx 0.309) — a genuinely randomized decision, illustrating why randomized tests are essential in the discrete case to hit an exact size. Its power under H1H_1 (p=0.6p=0.6): compute P1(X≥5)P_1(X\ge5) and P1(X=4)P_1(X=4) with p=0.6p=0.6. Using P1(X=k)=(8k)(0.6)k(0.4)8−kP_1(X=k)=\binom8k (0.6)^k(0.4)^{8-k}: P1(X=4)=70(0.6)4(0.4)4=70×0.1296×0.0256=0.2322P_1(X=4) = 70(0.6)^4(0.4)^4 = 70\times0.1296\times0.0256 = 0.2322, and cumulatively P1(X≥5)≈0.5941P_1(X\ge5) \approx 0.5941 (sum of k=5,…,8k=5,\dots,8 terms: 0.2787+0.1239+0.0413+0.0079≈0.45180.2787+0.1239+0.0413+0.0079\approx 0.4518; recomputing carefully, P1(X=5)=56(0.6)5(0.4)3=56×0.07776×0.064=0.2787P_1(X=5)=56(0.6)^5(0.4)^3=56\times0.07776\times0.064=0.2787, P1(X=6)=28(0.6)6(0.4)2=28×0.046656×0.16=0.2090P_1(X=6)=28(0.6)^6(0.4)^2=28\times0.046656\times0.16=0.2090, P1(X=7)=8(0.6)7(0.4)=8×0.0279936×0.4=0.0896P_1(X=7)=8(0.6)^7(0.4)=8\times0.0279936\times0.4=0.0896, P1(X=8)=0.68=0.01680P_1(X=8)=0.6^8=0.01680; sum =0.2787+0.2090+0.0896+0.0168=0.5941=0.2787+0.2090+0.0896+0.0168=0.5941). So

power=0.5941+0.3088×0.2322≈0.5941+0.0717=0.6658.\text{power} = 0.5941 + 0.3088\times 0.2322 \approx 0.5941+0.0717 = 0.6658.

Thus about 66.6%66.6\% power at exact size 10%10\% — again unimprovable by the lemma among size-0.100.10 tests for this problem.

Counterexampledropping the "size exactly attainable / simple hypotheses" structure — a composite alternative breaks the conclusion

The Neyman–Pearson Lemma requires both hypotheses to be simple — each specifying one fixed distribution. Suppose instead we weaken this and let the alternative be composite:

H0:θ=0vs.H1:θ≠0,H_0 : \theta = 0 \qquad \text{vs.} \qquad H_1 : \theta \ne 0,

for X∼N(θ,1)X \sim N(\theta, 1), a single observation, at level α=0.05\alpha = 0.05. If we naively try to apply the lemma "pointwise" by picking some particular θ1\theta_1, say θ1=3\theta_1 = 3, the likelihood ratio Λθ(x)=exp⁡(θx−θ2/2)\Lambda_\theta(x) = \exp(\theta x - \theta^2/2) is increasing in xx for θ1=3>0\theta_1 = 3 > 0, giving the one-sided most powerful test "reject if x>1.645x > 1.645" (using the 9595th percentile of N(0,1)N(0,1)). This test is optimal against θ1=3\theta_1=3 specifically, with power

Pθ=3(X>1.645)=P(Z>1.645−3)=P(Z>−1.355)≈0.9123.P_{\theta=3}(X > 1.645) = P(Z > 1.645 - 3) = P(Z > -1.355) \approx 0.9123.

But now test the same rejection region against a different alternative value, say θ1′=−3\theta_1' = -3 (equally consistent with H1:θ≠0H_1:\theta\ne0):

Pθ=−3(X>1.645)=P(Z>1.645+3)=P(Z>4.645)≈0.0000017,P_{\theta=-3}(X > 1.645) = P(Z > 1.645+3) = P(Z>4.645) \approx 0.0000017,

essentially zero power. Meanwhile the two-sided test "reject if ∣x∣>1.96|x|>1.96" — which is not of the likelihood-ratio form for any single alternative — has power Pθ=−3(∣X∣>1.96)≈P(Z<−1.96+3)P_{\theta=-3}(|X|>1.96) \approx P(Z < -1.96+3)-ish computations giving substantial power against both θ=3\theta=3 and θ=−3\theta=-3 symmetrically, roughly 0.850.85 against each, while the one-sided test derived from a single θ1\theta_1 is powerless against alternatives on the other side. There is consequently no single most powerful test of size 0.050.05 that dominates all others uniformly over θ≠0\theta \ne 0: the test optimal for θ1=3\theta_1=3 is nearly powerless for θ1=−3\theta_1=-3, and vice versa, so the conclusion "there exists a single test maximizing power against the alternative" fails once the alternative is composite. This is precisely why the theory of Likelihood Ratio Tests and uniformly most powerful tests requires extra structure (such as monotone likelihood ratio in one-sided problems) to recover any optimality once simplicity of H1H_1 is dropped.

Scope, edge cases & common traps

SituationWhat happens
Both H0,H1H_0, H_1 simple, continuous dataLemma applies directly; usually no randomization needed since likelihood ratio has a continuous null distribution.
Both H0,H1H_0, H_1 simple, discrete dataRandomization at the boundary {f1=kf0}\{f_1 = kf_0\} is generally required to hit exact size α\alpha.
f0(x)=0f_0(x) = 0 but f1(x)>0f_1(x) > 0 at some xxSuch xx automatically has Λ(x)=∞>k\Lambda(x)=\infty > k; include it in the rejection region; power there costs zero size.
Composite H1H_1Lemma does not directly apply; need monotone likelihood ratio (MLR) structure for a uniformly most powerful test, or move to Likelihood Ratio Tests / generalized approaches.
α=0\alpha = 0 or α=1\alpha = 1Degenerate: α=0\alpha=0 forces ϕ∗≡0\phi^*\equiv0 a.e.-P0P_0 (if P0,P1P_0,P_1 not mutually singular) giving zero power in typical cases; α=1\alpha=1 trivially allows ϕ∗≡1\phi^*\equiv1.

The mathematician behind it

Origins · 1 minJerzy Neyman and how this was discovered
JN
Jerzy Neyman
1894–1981 · Statistics, Warsaw / University College London / UC Berkeley

Jerzy Neyman was a Polish–American statistician who, together with Egon Pearson, built the modern frequentist theory of hypothesis testing in a series of papers beginning in 1928. Trained in Warsaw and Kharkiv in mathematics, Neyman shifted toward statistics after encountering Karl Pearson's work in London, then collaborated closely with Egon Pearson (Karl's son) at University College London through the late 1920s and 1930s. Their joint program replaced Ronald Fisher's significance-testing framework — which lacked an explicit alternative hypothesis — with a decision-theoretic structure comparing two specified hypotheses and formally optimizing test performance. Neyman later emigrated to the United States in 1938, founding the influential Statistics Laboratory at UC Berkeley, where he trained generations of statisticians and contributed foundational work on confidence intervals, survey sampling, and epidemiology.

History
- 1894 — Jerzy Neyman born in Bendery, Russian Empire (now Moldova).
- 1926 — Neyman begins correspondence and collaboration with Egon Pearson at UCL.
- 1928 — Neyman and Pearson publish the first joint paper introducing Type I/II errors and the likelihood-ratio comparison idea.
- 1933 — "On the Problem of the Most Efficient Tests of Statistical Hypotheses" (Philosophical Transactions of the Royal Society A) states and proves the Neyman–Pearson Lemma in its modern form.
- 1981 — Jerzy Neyman dies in Berkeley, California, having shaped the statistical curriculum worldwide.
What exactly does "most powerful" mean, and why fix the size first?

"Most powerful" means maximizing the probability of correctly rejecting H0H_0 when H1H_1 is true (the power), among all tests satisfying a constraint on the Type I error rate. Fixing the size first is necessary because power and size trade off against each other: a test that always rejects has power 11 but size 11, which is useless. By capping the false-positive probability at α\alpha and then asking for the best power subject to that cap, the problem becomes a genuine constrained optimization with a meaningful, non-trivial answer — exactly the setup the lemma solves via the likelihood ratio threshold.

Does the Neyman–Pearson Lemma apply when the hypotheses involve unknown parameters, like testing μ > 0 vs μ ≤ 0?

Not directly. The lemma as stated requires both H0H_0 and H1H_1 to be simple, each pinning down one exact distribution. When H1H_1 is composite (a range of parameter values), the most powerful test generally depends on which specific alternative you pick, as the counterexample shows. In special cases with a monotone likelihood ratio in a single parameter, one can still find a uniformly most powerful (UMP) test that works for every alternative in a one-sided composite hypothesis, but this requires additional structural assumptions beyond the lemma itself, leading into the broader theory of Likelihood Ratio Tests.

Why does the optimal test depend only on the likelihood ratio, and not on the raw data directly?

The proof shows that any deviation from thresholding the likelihood ratio can only lose power for a given size budget, because of the pointwise inequality (ϕ∗−ϕ)(f1−kf0)≥0(\phi^*-\phi)(f_1-kf_0)\ge 0. Intuitively, each observation xx "costs" f0(x)f_0(x) probability of Type I error and "earns" f1(x)f_1(x) probability of power if included in the rejection region; the ratio f1(x)/f0(x)f_1(x)/f_0(x) is exactly the earning rate per unit cost, so a rational allocator facing a fixed budget always spends on the highest earning-rate points first — which is precisely sorting by the likelihood ratio.

What role does randomization play, and is it ever actually used in practice?

Randomization is a mathematical device to hit an exact significance level when the test statistic's null distribution is discrete, creating an atom exactly at the critical value. In practice, most applied testing uses continuous statistics (t, F, chi-squared), so randomization is rarely implemented literally; instead, practitioners report a conservative test (size strictly less than the nominal α\alpha) or use exact p-values. However, in some discrete settings — Fisher's exact test, sequential clinical trial designs, or theoretical exercises — randomized tests are explicitly constructed to match the theory precisely.

How does the Neyman–Pearson Lemma relate to the Likelihood Ratio Test used in general (composite hypothesis) settings?

The general likelihood ratio test (LRT) for composite hypotheses computes sup⁡θ∈H0f(x;θ)\sup_{\theta\in H_0} f(x;\theta) divided into sup⁡θ∈Θf(x;θ)\sup_{\theta\in \Theta} f(x;\theta) and rejects for small values of this ratio; it is a direct generalization inspired by the simple-hypothesis case. When both the null and a candidate alternative reduce to single points, the general LRT reduces exactly to the Neyman–Pearson likelihood ratio, and the lemma's optimality (for that specific pair) still applies locally. But the general LRT for composite hypotheses is not proven optimal in the same strong sense; it is instead justified asymptotically (via Wilks' theorem) or via case-by-case analysis, making the simple-hypothesis lemma the exact, finite-sample ancestor of the more heuristic composite-case procedure.

Can the Neyman–Pearson framework be asymmetric in how it treats the two hypotheses, and is that a limitation?

Yes — the framework is deliberately asymmetric: it fixes the Type I error rate as a hard constraint and treats Type II error (or power) as the quantity to optimize, reflecting a "presumption of innocence" for H0H_0 common in scientific practice. This is a modeling choice, not a mathematical necessity; one could just as easily fix the Type II error and optimize Type I error, or seek a Bayes-optimal test balancing both under a loss function and prior. The lemma's asymmetric convention has become the near-universal default in classical statistics, but Bayesian and minimax alternatives handle the two error types more symmetrically.

What is the geometric or "ROC curve" interpretation of the lemma?

Plotting power (true positive rate) against size (false positive rate) as the threshold kk varies traces out the receiver operating characteristic (ROC) curve for the pair (P0,P1)(P_0, P_1). The Neyman–Pearson Lemma states that among all tests of a given size α\alpha (a given point on the horizontal axis), the likelihood-ratio test achieves the maximal power (the highest point on the vertical axis) — in other words, the likelihood-ratio family traces out exactly the upper boundary (concave envelope) of the achievable (size, power) region, and no test can lie strictly above this curve.

Before this

Where this leads