The Bonferroni Correction
Imagine screening 20,000 genes for a difference in expression between tumor and healthy tissue. Each single gene test uses α = 0.05, a 5% chance of a false positive. Run 20,000 of these tests. Roughly 1,000 genes will look "significant" purely by chance. Without a fix, your gene list is mostly noise dressed up as discovery. This is the central problem the Bonferroni correction was built to solve. It asks a simple question. If I accept only a 5% chance of any false positive across a study, how strict must each test become? The answer is stunningly simple arithmetic — divide α by the number of tests. That simplicity is exactly why the method has survived a century of statistical fashion. The Bonferroni correction matters because modern biology is a multiple-testing machine. Genome-wide association studies test millions of genetic variants. Microarray and RNA-seq experiments test thousands of genes. Even a modest ecology experiment comparing ten traits across five populations generates fifty tests. Any field that generates long lists of p-values needs a principled way to decide which hits are real. The Bonferroni correction is the oldest, strictest, and most intuitive member of that toolkit. This article builds the correction from first principles. It covers how the family-wise error rate arises. It also covers the algebra that controls it. It explains why the method is celebrated for its rigor. It also explains why it is criticized for its conservatism. It works through the classical logic (a Boole/Bonferroni inequality argument, not a single named "crucial experiment" in the Meselson–Stahl sense, since this is a statistical derivation, not a biological discovery). Then it walks through two fully worked numerical examples. It addresses the misconceptions that trip up nearly every student. Finally, it previews the modern alternatives — the False Discovery Rate and the statistics used in Genome-Wide Association Studies — that biology has largely adopted in its place.
- Testing many hypotheses inflates the chance of at least one false positive — the family-wise error rate (FWER).
- The Bonferroni correction controls FWER by testing each hypothesis at α/m, where m is the number of tests.
- It rests on a simple probability inequality (Boole's inequality), true regardless of dependence among tests.
- It guarantees FWER ≤ α but becomes very conservative as m grows large, inflating the false-negative rate.
- Genomic studies with thousands to millions of tests often replace it with the less strict False Discovery Rate.
- It remains the standard, required threshold in Genome-Wide Association Studies (e.g., p < 5 × 10⁻⁸).
To keep the chance of any false discovery across many tests below α, test each individual hypothesis at the much stricter threshold α divided by the number of tests.
The core idea
When a single hypothesis test is run at significance level α = 0.05, there is a 5% probability of rejecting a true null hypothesis — a Type I error, or false positive. This is the per-comparison error rate. Problems appear the moment more than one test is run on the same dataset or research question. If m independent tests are each conducted at α = 0.05, the probability that at least one produces a false positive is no longer 5%; it climbs toward 1 as m grows. This study-wide quantity is the family-wise error rate (FWER):
The multiple-testing problem is precisely this inflation of FWER. The Bonferroni correction, introduced into statistical practice by Olive Jean Dunn in 1961 (building on the 1930s inequality work of Carlo Emilio Bonferroni), is the most direct fix: instead of testing each hypothesis at the nominal α, test each one at
Equivalently, one can leave the threshold at α and instead multiply every raw p-value by m (capping at 1) before comparing to α. Both operations are algebraically identical and both guarantee that FWER never exceeds α, no matter how the tests are correlated with one another — a property that gives the method its enduring reputation for rigor. The price for that guarantee is conservatism: because the correction assumes the worst case (independent tests, each capable of contributing a false positive), it over-corrects whenever tests are positively correlated, and it shrinks the threshold so much at large m that true effects routinely fail to clear the bar. Understanding this trade-off — airtight control of false positives purchased with a rising rate of false negatives — is the entire pedagogical point of the method.
Mechanism
Step 1 — State the family-wise target. Choose an overall error budget α (conventionally 0.05): the maximum acceptable probability that the entire study produces one or more false positives.
Step 2 — Count the tests. Determine m, the number of independent hypotheses being tested simultaneously — e.g., m = 20,000 genes, or m = 500,000 single-nucleotide polymorphisms (SNPs).
Step 3 — Derive the per-test threshold using Boole's inequality. For any collection of events A₁, A₂, …, Aₘ (here, "test i produces a false positive"), probability theory guarantees:
This holds regardless of dependence among the Aᵢ — it needs no assumption that tests are independent. If each test's own false-positive probability is set to α/m, the sum on the right becomes m × (α/m) = α. Therefore:
The FWER is capped at α by construction. This is the entire mathematical content of the Bonferroni correction — one inequality, no calculus, no simulation.
Step 4 — Apply the threshold gene-by-gene (or SNP-by-SNP, or trait-by-trait). Each individual test's p-value is compared to α/m. A result is declared significant only if:
Step 5 — Interpret conservatism. Because Boole's inequality is an upper bound, not an exact equality, the true FWER achieved is often well below α — the method "wastes" error budget it didn't need to spend, especially when tests are positively correlated (as neighboring SNPs in Linkage Disequilibrium often are, since they are inherited together and are statistically far from independent). The correction does not know this and corrects as if every test were an independent chance to fail, so power — the probability of detecting a real effect — drops sharply as m increases. The figure below shows how the required per-test significance threshold shrinks as the number of tests grows, and how that shrinkage differs from the family-wise error actually achieved.
The next figure lays out the logical machinery: how many single tests combine into a family, how each false-positive probability is bounded, and how the sum caps the overall FWER.
The evidence
The Bonferroni correction is not established by a single laboratory experiment in the way Meselson and Stahl confirmed semi-conservative DNA replication; it is a mathematical guarantee, proven from probability axioms, and its "evidence" is the proof itself plus decades of applied validation in biology showing what happens when it is used — and what happens when it is not.
The mathematical proof (Bonferroni, 1936; applied to hypothesis testing by Dunn, 1961). Carlo Emilio Bonferroni's 1936 papers on probability established a family of inequalities bounding the probability of a union of events. Olive Jean Dunn, working at UCLA, recognized in 1959–1961 that these inequalities solved the simultaneous-inference problem: if each of m tests controls its own error at α/m, the union of "at least one is wrongly rejected" is bounded by α, exactly as derived above. Because the proof relies only on the general subadditivity of probability (P(A ∪ B) ≤ P(A) + P(B) for any events, correlated or not), it needed no simulation or empirical trial to be accepted — it is deductively certain. This is the "experiment" in the sense that matters here: a rigorous derivation that has never been overturned, only supplemented by less conservative alternatives.
The empirical demonstration of the problem it solves. The clearest empirical illustration of why the correction is needed comes from a deliberately absurd but instructive study: neuroscientist Craig Bennett and colleagues (2009, satirically titled "Neural correlates of interspecies perspective taking in the post-mortem Atlantic Salmon") performed a functional MRI scan on a dead salmon and ran the standard whole-brain voxel-wise statistical analysis, testing tens of thousands of brain voxels for activation without correcting for multiple comparisons. Several voxels showed "statistically significant" neural activity in a fish that was, unambiguously, deceased. This is Luria–Delbrück-style evidence not for a biological mechanism, but for a statistical one: with thousands of independent tests at uncorrected α = 0.001, spurious "activations" are essentially guaranteed. When the same salmon data were subjected to a proper multiple-testing correction (the study used a Bonferroni-style family-wise correction to make the point), the false "activity" disappeared. This demonstration became one of the most cited cautionary tales in neuroimaging statistics and is now a staple teaching example for exactly the failure mode the Bonferroni correction prevents.
Genomic validation. In early microarray studies of the 1990s–2000s, uncorrected testing of thousands of genes routinely produced gene lists in which a large fraction were later shown, on replication, to be false positives. Applying the Bonferroni correction (or its refinements) to these datasets sharply reduced the "significant gene" count but sharply increased the reproducibility of the surviving hits across independent cohorts — the pattern predicted by the theory: fewer discoveries, but a much higher fraction of them real. This trade-off, observed repeatedly across transcriptomic and genome-wide association datasets, is the practical confirmation that the correction behaves as its derivation predicts.
Worked examples
Example 1 — A gene-expression screen (asymmetric, realistic numbers). A lab compares gene expression between drought-stressed and well-watered maize plants using RNA-seq, testing m = 18,432 genes for differential expression, each with its own two-sample t-test. The lab wants overall FWER ≤ 0.05. Step 1: Bonferroni-adjusted threshold per gene:
Step 2: Suppose the raw p-values for five candidate genes are:
| Gene | Raw p-value | p < 2.71 × 10⁻⁶ ? |
|---|---|---|
| DREB2A (drought-response TF) | 8.4 × 10⁻⁸ | Yes — significant |
| RAB17 (LEA protein) | 3.9 × 10⁻⁶ | No — just misses threshold |
| ACT1 (actin, housekeeping) | 0.031 | No |
| P5CS1 (proline synthesis) | 1.2 × 10⁻⁷ | Yes — significant |
| CAT2 (catalase) | 4.0 × 10⁻⁵ | No |
Only DREB2A and P5CS1 survive. Note RAB17, despite an extremely small raw p-value that would be significant at the uncorrected α = 0.05 (or even α = 0.001) threshold, narrowly fails the corrected criterion — this is conservatism in action: a plausibly real drought-response gene is excluded because the correction must guard against 18,432 simultaneous chances for error. Reporting DREB2A and P5CS1 alone guarantees (assuming the statistical model is correctly specified) that there is at most a 5% chance that even one of the study's total gene calls is a false positive — a much stronger guarantee than 5% error on each gene individually.
Example 2 — A candidate-gene association study with unequal test counts (clinical scenario). A cardiology group tests whether any of 12 candidate SNPs in lipid-metabolism genes are associated with LDL cholesterol level in a cohort of 2,000 patients. They also, separately, run a secondary analysis testing the same 12 SNPs for association with 3 additional traits (HDL, triglycerides, and BMI), for a combined family of
if they choose to correct across the whole panel of traits (a defensible but conservative choice — some researchers would instead correct within each trait separately, using m = 12 four times, which is a less strict correction and a genuinely debatable analytic decision). Family-wise correction across all 48 tests:
Suppose the SNP rs693 (in APOB) yields:
| Trait | Raw p-value | Significant at 1.04×10⁻³? |
|---|---|---|
| LDL cholesterol | 2.0 × 10⁻⁴ | Yes |
| HDL cholesterol | 0.21 | No |
| Triglycerides | 0.045 | No (would pass uncorrected α=0.05!) |
| BMI | 0.88 | No |
Two lessons emerge. First, the LDL association survives even the stricter 48-test correction, giving strong confidence it is not a chance artifact of testing many traits. Second, the triglyceride association (p = 0.045) would have been called "significant" under a naïve single-test α = 0.05 threshold but is correctly rejected once the analyst accounts for the full family of comparisons actually performed — illustrating why the definition of "the family" (all 48 tests, or 12 within each trait) is itself a consequential scientific judgment call, not a mechanical formality.
Common misconceptions & traps
| Misconception | Why it's wrong |
|---|---|
| "Bonferroni correction changes the biology of the result." | It only changes the statistical threshold for declaring significance; the underlying effect size and biological reality are unaffected. |
| "A Bonferroni-corrected non-significant result proves there is no effect." | Failing to reach α/m is a failure to reject the null at a very strict standard — it is consistent with a real but underpowered effect, not proof of no effect. |
| "You must always divide by the total number of tests ever run in the lab." | The correction applies to the specific family of hypotheses relevant to one conclusion; defining that family (all genes in one screen? all traits across all screens?) is a judgment call, not automatic. |
| "The Bonferroni correction requires the tests to be independent." | It is valid for any dependence structure — that is precisely its mathematical strength, though it becomes especially conservative when tests are positively correlated. |
| "Because it's conservative, Bonferroni should never be used in genomics." | It remains the standard for Genome-Wide Association Studies, where an extremely low tolerance for false positives is deliberately prioritized over sensitivity. |
| "A p-value of 0.04 is 'basically significant' even after correction fails." | Under the corrected threshold, only p < α/m counts; there is no partial credit — the whole point of a family-wise guarantee is a firm, pre-specified cutoff. |
The scientist behind it
The correction bears the name of Carlo Emilio Bonferroni (1892–1960), an Italian mathematician working mostly at the University of Florence and University of Bologna, whose research area was probability theory and actuarial mathematics — not biology or genetics at all. His 1935 and 1936 papers established a family of inequalities bounding the probability of unions and intersections of events, generalizing what is now sometimes called "Boole's inequality" to higher orders. Bonferroni had no direct involvement in biological statistics; his inequalities sat in the probability literature for over two decades before anyone connected them to the practical problem of simultaneous hypothesis testing. That connection was made by Olive Jean Dunn (1915–2008), an American biostatistician at UCLA, whose 1959 and 1961 papers ("Estimation of the medians for dependent variables" and "Multiple Comparisons Among Means") formally applied Bonferroni's inequality to the problem of constructing simultaneous confidence intervals and controlling error across multiple statistical comparisons. Dunn's work is why some texts more precisely call the technique the "Bonferroni–Dunn" correction, and why the method entered biology and medicine through applied statistics rather than through a biological discovery narrative.
Origins · 1 minCarlo Emilio Bonferroni and how this was discovered
Where this matters today
Genome-wide association studies (GWAS). The now-standard genome-wide significance threshold of p < 5 × 10⁻⁸ is a Bonferroni correction: it approximates α = 0.05 divided by roughly one million independent tests, reflecting the estimated number of effectively independent common variants across the human genome after accounting for Linkage Disequilibrium. Every major GWAS of height, diabetes risk, or psychiatric disorders reports hits against this threshold, foreshadowing the more detailed treatment in Genome-Wide Association Statistics.
Clinical trials with multiple endpoints. Regulatory agencies (FDA, EMA) frequently require Bonferroni or related corrections when a trial tests a drug against several outcomes (e.g., survival, tumor shrinkage, and quality of life) to prevent a sponsor from cherry-picking one "significant" endpoint out of several tested.
Diagnostic and screening panels. Multi-gene cancer panels (e.g., testing dozens of genes for pathogenic variants in hereditary cancer risk) apply family-wise corrections to avoid flagging a patient's variant as clinically significant based on chance alone.
Agricultural and ecological field trials. Breeding programs testing many crop varieties across many environmental conditions, or ecologists testing multiple species-by-site combinations for a treatment effect, use Bonferroni-style corrections to avoid declaring spurious variety-by-environment interactions.
Conservation genetics. Studies scanning many microsatellite or SNP loci for signatures of selection or population differentiation apply the correction (or its relatives) to avoid mistaking sampling noise for a genuine adaptive locus, an important safeguard before recommending conservation management changes based on a single flagged marker. In all these settings, the same tension recurs: Bonferroni's ironclad guarantee against false positives is purchased with reduced sensitivity, which is exactly why genomics has increasingly turned toward the False Discovery Rate framework — controlling the expected proportion of false discoveries among rejected hypotheses rather than the probability of any false discovery at all — as a less conservative, often more practically useful standard for very large-scale biological screens.
Ask the tutor: How does the choice between controlling FWER (Bonferroni) versus controlling the False Discovery Rate change which genes a genomic study would report as significant?
Why divide by the number of tests instead of some other adjustment?
Dividing by m is the direct algebraic consequence of Boole's inequality: if each of m tests is allowed a false-positive probability of α/m, the sum of those probabilities — which upper-bounds the probability that at least one test is a false positive — works out to exactly m × (α/m) = α. No other simple per-test adjustment gives this guarantee for arbitrary dependence structures. More refined methods (Holm's step-down procedure, Šidák's correction) tighten the bound slightly using additional assumptions, but the basic division by m is the simplest choice that provably achieves FWER ≤ α under all conditions.
Does the Bonferroni correction work if my tests are correlated, like neighboring SNPs in linkage disequilibrium?
Yes — the correction's FWER ≤ α guarantee holds regardless of correlation, because Boole's inequality makes no independence assumption. However, correlation among tests means the bound is looser than necessary: the true FWER achieved is often much lower than α, making the correction even more conservative than the independent-tests case. This is exactly why GWAS analysts sometimes estimate the "effective number of independent tests" (accounting for linkage disequilibrium) rather than using the raw SNP count, arriving at thresholds like 5 × 10⁻⁸ instead of an even stricter value based on millions of raw genotyped markers.
What's the difference between the Bonferroni correction and the Šidák correction?
Both control FWER, but the Šidák correction assumes the tests are independent and uses the formula α_adj = 1 − (1 − α)^(1/m), which is very slightly less conservative than α/m. For typical values of m and α, the two thresholds are numerically almost identical (e.g., at m = 20 and α = 0.05, Bonferroni gives 0.0025 and Šidák gives 0.00256), so the practical difference is small. Bonferroni remains more popular because it needs no independence assumption and its formula is simpler to compute and explain.
Why do genomics researchers often prefer the False Discovery Rate over the Bonferroni correction?
The Bonferroni correction controls the probability of any false positive across the whole study, which becomes extremely strict — and correspondingly low-powered — when m reaches the tens or hundreds of thousands typical of transcriptomic or genomic data. The False Discovery Rate instead controls the expected proportion of false positives among all findings called significant, tolerating some false positives as long as most discoveries are real. This trade-off suits exploratory genomic screens, where researchers plan to validate top hits experimentally anyway and prioritize sensitivity over an ironclad zero-tolerance guarantee.
Can the Bonferroni correction be applied to a single test, or only to multiple tests?
It is meaningless for a single, pre-specified test — with m = 1, α/m = α/1 = α, so the correction has no effect. Its logic only engages once more than one hypothesis is tested and a study-wide guarantee is desired. Applying "a Bonferroni correction" to a solitary confirmatory test (as sometimes seen in poorly designed papers) reflects a misunderstanding of what family-wise error even means, since there is no "family" of comparisons to protect against.
How do I decide what counts as "the family" of tests to correct across?
This is a genuine judgment call rather than a formula, and it is one of the most common places analyses go wrong. A defensible rule of thumb is: the family should include every test whose results could plausibly be cherry-picked to support the same overarching scientific claim. If a paper tests 20,000 genes for one comparison (treatment vs. control) and reports the top hits as "genes responding to treatment," the family is those 20,000 tests. If the same lab separately tests those genes for a completely different comparison in a different figure with its own claim, many statisticians treat that as a separate family, though pooling everything into one enormous correction is always more conservative and defensible if reviewers demand it.
Is the Bonferroni correction ever "too conservative to be useful"?
Yes, and this is well documented in fields like whole-genome methylation studies or untargeted metabolomics, where m can exceed 10⁶ and Bonferroni thresholds fall below 10⁻⁸ — a level that only extremely large or extremely strong effects can ever reach, even when many weaker true effects exist. In such settings researchers commonly report Bonferroni results as the strict "gold-standard" hits while using the False Discovery Rate as the primary discovery tool, treating the two thresholds as complementary rather than choosing only one.