Inbreeding and the Inbreeding Coefficient

Inbreeding is mating between individuals who are more closely related than two individuals chosen at random from the population. Every finite population has some background relatedness—given enough generations, any two individuals share ancestors—but "inbreeding" in the genetic sense refers to matings between identifiably close relatives: first cousins, siblings, parent-offspring, or, in conservation and agricultural genetics, any mating within a small closed population where relatedness accumulates. The central quantity is the inbreeding coefficient, denoted F. Formally, F is the probability that the two alleles at a given autosomal locus in an individual are identical by descent (IBD)—that is, both are physical copies of a single allele carried by a common ancestor, rather than merely identical in state (the same allele by coincidence, inherited independently). F is a property of an individual, defined relative to a specified base population or pedigree, and it applies uniformly to every locus in the genome (aside from statistical sampling noise at any one locus), because it reflects a genome-wide pattern of shared ancestry, not something about any particular gene. It is essential to keep two effects distinct. First, inbreeding is a mating system effect: it changes genotype frequencies (more homozygotes, fewer heterozygotes) without altering allele frequencies at all—unlike Factors That Change Allele Frequencies such as selection, drift, migration, or mutation, which do change allele frequencies. Second, layered on top of this genotypic shift is a phenotypic consequence: inbreeding depression, the reduction in fitness-related traits (fertility, survival, growth rate, resistance to disease) that tends to accompany increased homozygosity, because it unmasks rare recessive deleterious alleles and disrupts favorable heterozygous combinations (overdominance). Inbreeding itself is a demographic/mating phenomenon; inbreeding depression is its typical evolutionary and phenotypic consequence.

From pedigrees to probabilities: what F actually counts

Consider an individual X whose parents are related through one or more common ancestors. F for X is computed by tracing every path through the pedigree that connects X's father to X's mother via a shared ancestor, and summing the probability that an allele is transmitted unchanged along each path. The rule, formalized by Sewall Wright in the 1920s, is:

summed over every distinct path connecting the two parents through a common ancestor A, where n is the number of links (individuals) in that path connecting one parent to the other through A, and F_A is the inbreeding coefficient of the common ancestor itself (usually taken as 0 if A is assumed non-inbred/founder). Each "link" in the path represents one meiosis, and at each meiosis there is a 1/2 chance that a particular allele is transmitted. A path through a common ancestor represents one specific route by which the same physical allele copy in ancestor A could reach both parents and then be transmitted to both of X's gametes that unite at fertilization.

Worked mini-derivation for first cousins. Suppose X's parents are first cousins, sharing two grandparents (G1 and G2) as common ancestors, and assume G1 and G2 are themselves non-inbred (F=0). Path through G1: father's mother → father → X, and G1 → mother's mother → mother → X. Counting links: G1 → (child of G1) → father → X is 3 links on one side, and G1 → (child of G1) → mother → X is 3 links on the other side; combined path length n = 3 + 3 = 6 (six meioses connecting the two parents through G1). Contribution from G1: (1/2)^6 = 1/64. Contribution from G2: (1/2)^6 = 1/64.

This is the textbook value: children of first cousins have F = 1/16. For comparison, offspring of full siblings have F = 1/4, offspring of half-siblings F = 1/8, and offspring of double first cousins (sharing all four grandparents) F = 1/8.

What F does to genotype frequencies

Let p and q be the population allele frequencies for a biallelic locus (A and a), as usual under The Hardy-Weinberg Principle. Without inbreeding, genotype frequencies are p², 2pq, q². With inbreeding coefficient F, a fraction F of individuals are "autozygous"—their two alleles are IBD copies of a single ancestral allele—and are therefore automatically homozygous (AA with probability p, aa with probability q, in proportion to allele frequency). The remaining fraction (1−F) mate as if randomly, giving ordinary Hardy-Weinberg proportions. Combining:

Heterozygosity is reduced by a factor (1−F) relative to Hardy-Weinberg expectation:

This single equation is the mechanistic heart of the topic: F is literally defined as the fractional reduction in heterozygosity relative to random-mating expectation. Notice that allele frequencies p and q never change in these formulas—only their partitioning among genotypes shifts. This is the crucial distinction from selection or drift, which move p and q themselves.

Why homozygosity causes depression: unmasking recessive load

Most populations carry a "genetic load" of rare deleterious recessive alleles maintained at low frequency by mutation-selection balance, hidden almost entirely in heterozygous carriers because selection cannot "see" a recessive allele unless it is homozygous. Inbreeding, by increasing homozygosity uniformly across the genome, systematically converts this hidden heterozygous load into expressed homozygous load. Two complementary genetic mechanisms are usually invoked: 1. Partial dominance (directional dominance) hypothesis: most deleterious mutations are recessive or partially recessive; inbreeding exposes them as homozygotes, lowering fitness. This is now supported as the dominant explanation in most systems. 2. Overdominance (heterozygote advantage) hypothesis: at some loci the heterozygote itself is fittest (as in sickle-cell/malaria); inbreeding reduces the frequency of these superior heterozygotes directly, independent of any deleterious recessive being unmasked. Both mechanisms predict the same qualitative outcome—reduced mean fitness with increasing F—but they differ in predicted response to purging (removal of deleterious alleles by selection during sustained inbreeding) and in expected genetic architecture; modern molecular and quantitative genetic evidence favors partial dominance as the primary driver in most species, with overdominance playing a smaller but real role at specific loci.

Effective population size and the accumulation of F over generations

In a finite, randomly mating population of effective size Nₑ, F does not stay at zero—every generation of finite population size adds a small increment of unavoidable inbreeding, because in a small population any two gametes have some chance of tracing to the same parent. The classic recursion is:

This shows why small populations (a captive breeding program, an island population, an endangered species reduced to a handful of founders) inevitably accumulate inbreeding over time even without deliberate mating of known relatives—random mating in a small closed population is itself a form of unavoidable inbreeding. This links directly to genetic drift as a companion force: both are consequences of finite population size, but drift describes changes in allele frequency variance while F describes the resulting increase in homozygosity within lineages.

Sewall Wright's pedigree method and guinea pig experiments (1910s–1920s)

The concept of F and its systematic calculation from pedigrees was developed by Sewall Wright, working first at the USDA and then studying guinea pigs bred at the US Department of Agriculture's experiment station in Bethesda, Maryland. Wright had inherited decades of meticulous breeding records from earlier USDA geneticist workers, including highly inbred lines produced by repeated brother-sister mating over 20+ generations, alongside outbred control lines. Wright's key innovation (published 1921–1922) was the method of path coefficients, which allowed him to compute F for any individual from an arbitrarily complex pedigree by summing contributions over all paths through common ancestors—the same formula given above. He then correlated calculated F values with measured phenotypes across the guinea pig lines.

What he found: inbred lines (F approaching 0.9–1.0 after many generations of sib-mating) showed, compared to outbred controls:

• Substantially reduced litter size and increased stillbirth rate
• Reduced body weight and slower growth
• Increased susceptibility to disease
• Considerable variation between different inbred lines—some inbred lines were nearly as vigorous as outbred stock, while others were severely compromised, showing that different lines had become homozygous for different combinations of deleterious recessives (consistent with the partial dominance hypothesis, since random fixation of different bad alleles in different lines explains why some lines suffered more than others).

What it proved: This was the first rigorous, quantitative demonstration that a calculable pedigree-based coefficient (F) predicts phenotypic decline (inbreeding depression) in a dose-dependent way, and that the severity of decline is not fixed but depends on which particular deleterious alleles happen to become fixed—directly supporting the idea that inbreeding depression results from unmasking recessive deleterious variants rather than from some intrinsic harm of homozygosity itself.

Drosophila viability studies and quantifying genetic load

Following Wright, geneticists including Theodosius Dobzhansky and colleagues used Drosophila melanogaster to make the recessive-lethal hypothesis directly testable at the chromosomal level. Using balancer chromosome techniques (chromosomes carrying multiple inversions that suppress recombination and carry visible dominant markers, developed by Hermann Muller), researchers could make individual wild chromosomes homozygous in controlled crosses and directly score the fraction that were lethal or sub-vital when homozygous.

What was found: a substantial fraction of wild-collected second and third chromosomes (historically found to be roughly 20–50% depending on population and chromosome, with results varying across studies) were lethal or semi-lethal when made homozygous, despite the flies carrying them being perfectly healthy as heterozygotes in the wild. This directly demonstrated that natural, outbred populations carry a large hidden load of recessive deleterious alleles—the exact raw material that inbreeding converts into expressed homozygous phenotypes.

What it proved: it moved the explanation for inbreeding depression from a statistical inference (Wright's guinea pig correlations) to a mechanistic demonstration—showing directly, chromosome by chromosome, that heterozygosity really was masking a large recessive genetic load, and that forcing homozygosity (exactly what inbreeding does at the population level) reliably reveals it.

Human consanguinity studies

Independent evidence comes from human populations with high rates of consanguineous marriage (marriage between relatives, most commonly first cousins in many parts of the Middle East, South Asia, and North Africa). Large epidemiological studies—for example, cohort studies in Pakistan, the Middle East, and historical parish-record studies in France, Sweden, and Japan—compared offspring of first-cousin marriages (F = 1/16) to offspring of unrelated parents.

What was found: consistent, replicated increases in:

What it proved: even a modest F (1/16, far below Wright's highly inbred guinea pig lines) produces statistically detectable, reproducible elevations in mortality and recessive disease incidence in a large, genetically heterogeneous species (humans), confirming that inbreeding depression is not an artifact of small laboratory populations but a general genetic phenomenon whose magnitude scales, as predicted, with F.

Example 1: Calculating F for uncle-niece mating and predicting albinism risk

A population has albinism (an autosomal recessive disorder, genotype aa) at allele frequency q = 0.01 (so p = 0.99). A man mates with his niece (his brother's daughter). Assume no other relatedness and that founding ancestors are non-inbred.

Step 1 — find F for the offspring. Uncle-niece is a well-known relationship with a single class of connecting path: the niece's father (the uncle's brother) is the key intermediate. The two full-sibling parents (the uncle and the niece's father) share two common ancestors, the niece's grandparents (call them GP1, GP2). Paths for uncle-niece offspring:

• Path through GP1: uncle → GP1 → brother → niece → child. Counting links from one "parent" (uncle) to the other "parent" (niece) through GP1: uncle–GP1 (1 link) + GP1–brother (1 link) + brother–niece (1 link) = 3 links, so n = 3.
• Same for GP2: n = 3.

(Working through the arithmetic explicitly: (1/2)³=1/8 for each of the two grandparents, total F = 1/8+1/8 = 1/4... but the standard, well-established value for uncle-niece/aunt-nephew matings is F = 1/8. The discrepancy above stems from an easy path-counting error; the correct link count for uncle-niece is n = 4 per path (uncle→GP→brother→niece is 3 links, but the standard convention counts the number of individuals in the connecting path excluding the common ancestor itself in one specific way that yields n=4), giving (1/2)⁴+(1/2)⁴ = 1/16+1/16 = 1/8.) We adopt the standard, textbook-verified result:

Step 2 — genotype frequencies in the offspring, using p = 0.99, q = 0.01, F = 0.125:

Step 3 — compare to the random-mating baseline. Without inbreeding, freq(aa) = q² = 0.0001 = 1/10,000. With F = 1/8, freq(aa) ≈ 0.00134 ≈ 1/747.

Interpretation: the risk of albinism in this child is roughly 13-fold higher than in the general population, even though the allele frequency q = 0.01 is unchanged in the population at large. This is the mechanism in miniature: no allele frequency changed, but a rare recessive became far more likely to appear in homozygous, affected form because of shared ancestry. This is exactly why genetic counselors quote elevated relative risks for consanguineous couples even when the underlying allele is individually rare.

Example 2: A captive-breeding population and the erosion of heterozygosity

A conservation program starts a captive population of an endangered antelope with effective population size Nₑ = 25 and maintains random mating within this closed population (no further founders added) for 10 generations.

Step 1 — apply the recursion $F_t = \frac{1}{2N_e} + (1-\frac{1}{2N_e})F_{t-1}$, with F₀ = 0 and 1/(2×25) = 0.02. This recursion is equivalent to:

Step 2 — interpret. After only 10 generations in this small closed population, the average inbreeding coefficient reaches roughly F ≈ 0.18, comparable to the offspring of half-siblings (F=0.125) or worse—purely from finite population size, with no deliberate mating of known relatives at all.

Step 3 — predict heterozygosity loss. Since H = H₀(1−F), the population is expected to have lost about 18% of its original heterozygosity at every locus genome-wide relative to the founding population.

Step 4 — practical consequence. If this species carries a genetic load similar to Wright's guinea pigs or Dobzhansky's Drosophila chromosomes, this level of F predicts a measurable decline in fertility, juvenile survival, and disease resistance—precisely the pattern documented in real endangered-species programs (e.g., Florida panthers, which showed kinked tails, cryptorchidism, and low sperm quality attributable to inbreeding depression before a deliberate genetic rescue via introduction of Texas cougars in 1995). This example shows why conservation genetics treats Nₑ, not just census population size, as the critical number to manage, and why genetic rescue (introducing unrelated individuals) is often used to reverse accumulated F.

Sewall Wright trained initially as an animal breeding specialist, working at the USDA on guinea pig genetics under the mentorship of William Ernest Castle at Harvard, before moving to Chicago in 1926 where he spent most of his career. Wright was one of the three founders (with R.A. Fisher and J.B.S. Haldane) of theoretical population genetics, but uniquely among them, his path into the field ran directly through decades of hands-on animal breeding records rather than pure mathematics. It was this practical grounding in real pedigrees—hundreds of guinea pigs, generations of matings, actual litter sizes and stillbirth counts—that led him to invent both the inbreeding coefficient F and the broader method of path analysis (later foundational to path coefficients in quantitative genetics and, independently, to structural equation modeling in statistics). He also developed the concept of genetic drift and the shifting balance theory of evolution, emphasizing how small, partially isolated populations (with elevated F) could explore adaptive landscapes differently than large panmictic populations—directly tying his inbreeding work to his broader evolutionary theorizing.

Clinical genetics and genetic counseling. Consanguinity assessment is now routine in prenatal and pediatric genetics; calculated F values (or genomic estimates of "runs of homozygosity," ROH, from SNP arrays) inform recurrence-risk counseling for recessive disorders in populations with high rates of cousin marriage, and clinical exome sequencing routinely screens ROH-rich regions of the genome for recessive disease-causing homozygous variants.

Livestock and crop breeding. Deliberate inbreeding is used to create genetically uniform inbred lines (in maize, for example), which are then crossed to produce F1 hybrids that display heterosis (hybrid vigor)—essentially inbreeding depression run in reverse, since crossing two different inbred lines restores heterozygosity at loci where each line is fixed for different alleles. This is the genetic basis of virtually all commercial hybrid maize seed sold worldwide, and analogous crossbreeding schemes are used in poultry, pig, and dairy cattle breeding to manage inbreeding depression while retaining desirable homozygous traits.

Conservation biology. Effective population size and accumulated F are now standard, mandatory metrics in species recovery plans. Documented cases—Florida panthers, Isle Royale wolves, Scandinavian wolf populations—show clear, measurable inbreeding depression (skeletal defects, reduced sperm quality, reduced litter survival), and "genetic rescue" via deliberate introduction of unrelated individuals has become a standard management tool once F crosses concerning thresholds.

Dog and purebred animal breeding. Kennel club breed standards historically enforced closed studbooks, driving F upward across many purebred dog lines over decades; this is now widely recognized as the genetic basis for the elevated incidence of hip dysplasia, heart disease, and various cancers in specific breeds, motivating outcrossing programs and genetic-diversity-based breeding recommendations.

Human population history and forensic genetics. Genome-wide estimation of individual F from SNP or whole-genome data (via detected runs of homozygosity) is used to reconstruct population history, detect recent bottlenecks, and, in forensic and immigration contexts, to assess biological relatedness between individuals.

Ask the tutor: How would you estimate an individual's inbreeding coefficient directly from genome sequencing data, without any pedigree information at all?