Inbreeding Depression

Inbreeding depression is the reduction in biological fitness—quantified across survival to sexual maturity, fecundity, growth rate, physiological vigor, and disease resistance—observed in the progeny of genetically related individuals compared to the progeny of unrelated outbred individuals within the same population. First investigated systematically across dozens of plant species by Charles Darwin in The Effects of Cross and Self Fertilisation in the Vegetable Kingdom (1876) and formalized mathematically by Sewall Wright, J. B. S. Haldane, and James F. Crow, inbreeding depression is a direct consequence of increased genome-wide homozygosity (autozygosity) resulting from biparental consanguinity, self-fertilization, or small effective population size (N_e).

Although inbreeding does not by itself alter allele frequencies (p and q) in the gene pool, it redistributes genotypes away from Hardy-Weinberg principle proportions by converting heterozygosity into homozygosity across every locus in the genome simultaneously. This elevation in homozygosity unmasks rare, deleterious recessive mutations that normally remain hidden from purifying natural selection inside heterozygous carriers, driving phenotypic decline and shaping the evolution of mating systems, incest avoidance behaviors, and plant self-incompatibility mechanisms.

Interactive Simulation: Track how inbreeding coefficients (F) rise in small populations and expose recessive genetic load in the Allele Drift Simulator.

Wright's Inbreeding Coefficient (F) and Identity by Descent

To quantify the degree of inbreeding in an individual or population, Sewall Wright (1922) defined the inbreeding coefficient (F), which Gustave Malécot (1948) reformulated in probabilistic terms as the probability that the two alleles carried by a diploid individual at any given autosomal locus are identical by descent (IBD) (autozygous)—meaning both copies are replication descendants of a single ancestral allele present in a common ancestor shared by the maternal and paternal lineages. The complementary fraction (1 − F) represents the probability that the two alleles are drawn independently from the ancestral gene pool (allozygous).

In a pedigree where a male and female share one or more common ancestors (A), the inbreeding coefficient of their offspring X is computed by Wright's path-coefficient formula:

F_X = ∑_A (1 / 2)^{n₁ + n₂ + 1} · (1 + F_A)

where n₁ is the number of generations from the sire back to common ancestor A, n₂ is the number of generations from the dam back to A, and F_A is the inbreeding coefficient of ancestor A itself. For example, self-fertilization produces F = 1/2 (0.50) in a single generation, full-sibling or parent-offspring mating produces F = 1/4 (0.25), half-sibling or double-first-cousin mating produces F = 1/8 (0.125), and first-cousin mating (two shared grandparents with n₁ = 2, n₂ = 2) yields F = 2 × (1/2)⁵ = 1/16 (0.0625).

Genotype Frequencies and Linear Decline of Mean Fitness

Consider a biallelic locus with wild-type allele A₁ at frequency p and deleterious mutant allele A₂ at frequency q = 1 − p. In a population with inbreeding coefficient F, a fraction F of zygotes is autozygous (forming A₁A₁ with probability p and A₂A₂ with probability q), while the remaining fraction (1 − F) combines alleles at random in Hardy-Weinberg proportions:

• Wild-type homozygote P(A₁A₁): p²(1 − F) + pF = p² + pqF
• Heterozygote P(A₁A₂): 2pq(1 − F)
• Mutant homozygote P(A₂A₂): q²(1 − F) + qF = q² + pqF

Assigning standard relative fitnesses w₁₁ = 1, w₁₂ = 1 − hs, and w₂₂ = 1 − s (where s > 0 is the selection coefficient against the mutant homozygote and h is the dominance coefficient), the mean fitness of the inbred population (w̄_F) is:

w̄_F = 1 − 2pqhs(1 − F) − s(q² + pqF) = w̄₀ − pq · s · (1 − 2h) · F

where w̄₀ = 1 − 2pqhs − q²s is the mean fitness of an outbred population (F = 0). Crucially, this equation demonstrates that inbreeding reduces mean population fitness (w̄_F < w̄₀) if and only if h < 1/2—that is, when deleterious alleles are fully or partially recessive (directional dominance). If deleterious effects were strictly additive across alleles (h = 1/2, so 1 − 2h = 0), increasing homozygosity would have zero effect on mean fitness because every gain of a double-mutant A₂A₂ homozygote would be exactly offset by the elimination of two partially impaired A₁A₂ heterozygotes.

The Partial Dominance (Recessive Mutation) Hypothesis

First proposed by Charles Davenport (1908) and formalized by Donald F. Jones (1917) and James F. Crow (1948, 1952), the partial dominance hypothesis attributes inbreeding depression to the unmasking of thousands of rare, partially recessive deleterious alleles (h ≈ 0.02–0.25) maintained across the genome at mutation-selection balance (q̂ ≈ μ / [hs] for partially recessive alleles, or q̂ = √[μ/s] for completely recessive alleles where h = 0). Because q is very small (typically 10⁻⁴ to 10⁻²), in an outbred population almost all copies of the deleterious allele reside harmlessly inside heterozygotes (2pq ≫ q²). Under inbreeding, however, the homozygous recessive frequency q²(1 − F) + qF ≈ qF scales linearly with q rather than q², inflating the incidence of homozygous recessive genetic disorders by orders of magnitude (as shown in the table above, where a first-cousin marriage increases the incidence of a recessive disease with q = 0.01 by more than 7-fold).

The Overdominance Hypothesis

Proposed independently by George H. Shull and Edward M. East (1908), the overdominance hypothesis posits that inbreeding depression arises from the loss of intrinsically superior heterozygotes at loci governed by heterozygote advantage (w₁₂ > w₁₁ and w₁₂ > w₂₂). If homozygotes have fitnesses w₁₁ = 1 − s and w₂₂ = 1 − t relative to w₁₂ = 1, the polymorphic equilibrium allele frequencies are p̂ = t / (s + t) and q̂ = s / (s + t), and inbreeding increases the segregation load linearly with F: w̄_F = 1 − st / (s + t).

Modern quantitative genomic and QTL mapping experiments across Drosophila, maize (Zea mays), and wild vertebrates have resolved this century-old debate: partial dominance of recessive deleterious alleles accounts for the vast majority (>85–95%) of genome-wide inbreeding depression, while true single-locus overdominance is restricted to specialized immune and recognition systems (such as the vertebrate Major Histocompatibility Complex) and apparent pseudo-overdominance (where two tightly linked loci carry deleterious recessives in repulsion phase, A₁b₂ / a₂B₁).

Morton-Crow-Muller Regression and Lethal Equivalents (B)

Assuming that deleterious loci act multiplicatively (independently) on survival probability S, Newton E. Morton, James F. Crow, and Hermann J. Muller (1956) derived the negative exponential survival relationship (exp(...)):

S = exp( −A − B · F ) ⇒ −ln(S) = A + B · F

Here, A represents baseline mortality in an outbred population (F = 0) due to environmental hazards and dominant genetic load, while the regression slope B = ∑ q_is_i(1 − 2h_i) defines the number of haploid lethal equivalents (or 2B per diploid genome). One haploid lethal equivalent is a set of dispersed recessive deleterious alleles that, if made completely homozygous (F = 1), would cause on average one lethal event per gamete (reducing survival to e⁻¹ ≈ 36.8%). Across wild bird and mammal populations, juvenile survival studies typically measure B ≈ 1.5 to 6.0 haploid lethal equivalents (3 to 12 per diploid zygote), whereas long-lived gymnosperms (such as Douglas fir and Scots pine) harbor B ≈ 8 to 15 lethal equivalents.

Genetic Purging vs Mutational Fixation

When a population undergoes slow, sustained inbreeding (such as selfing plants or gradual demographic contraction where N_es ≫ 1), the increased expression of homozygous recessive genotypes exposes deleterious alleles to purifying natural selection, allowing them to be selectively eliminated from the gene pool—a process called genetic purging. Purging is highly effective against fully recessive lethal or semi-lethal alleles (s → 1, h → 0), which explains why historically self-fertilizing plants (such as Arabidopsis thaliana) exhibit much lower inbreeding depression (δ = 1 − w_self/w_outcross < 0.20) than obligate outcrossers (δ > 0.50). However, purging fails against mildly deleterious alleles (s < 0.05, h ≈ 0.25) during rapid population bottlenecks because genetic drift overpowers weak selection, fixing deleterious alleles across the population and requiring incoming gene flow (genetic rescue) to restore fitness.

How does inbreeding differ from positive assortative mating?

Consanguineous inbreeding (mating between genealogical relatives) increases autozygosity (F) uniformly across every locus in the genome—both coding and neutral non-coding DNA—because related parents share identical chromosomal segments inherited from recent common ancestors. By contrast, positive assortative mating (mating between individuals with similar phenotypes, such as body size or flower color) increases homozygosity only at the specific loci controlling the selected trait and tightly linked regions, leaving unlinked neutral loci across the rest of the genome in Hardy-Weinberg proportions.

Why do some self-fertilizing plants and island species survive with high homozygosity and little inbreeding depression?

Lineages that evolve a long-term mixed or predominantly selfing mating system (or persist at small N_e for thousands of generations without going extinct, such as the Channel Island fox Urocyon littoralis) have survived an evolutionary filter of genetic purging. Over hundreds of generations of partial inbreeding, lethal and strongly deleterious recessive alleles (s > 0.1) were exposed in homozygotes and eliminated by natural selection, leaving behind a genome depleted of high-effect segregating recessive load (low B slope), even though weakly deleterious mutations may still be fixed as drift load.

What is heterosis (hybrid vigor), and how is it related to inbreeding depression?

Heterosis (or hybrid vigor) is the mirror-image phenomenon of inbreeding depression: when two distinct inbred lines (each having high F and fixed for different sets of mildly deleterious recessive alleles, e.g., line 1 is aaBBcc and line 2 is AAbbCC) are crossed, their F₁ hybrid offspring (AaBbCc) return to F = 0. Because the dominant wild-type allele from one parent masks the recessive deleterious allele from the other parent across all complemented loci (complementation under partial dominance), F₁ hybrids exhibit a dramatic rebound in growth rate, grain yield, and survival—forming the genetic foundation of modern hybrid maize agriculture.

1. Charlesworth, D., & Willis, J. H. (2009). "The genetics of inbreeding depression." Nature Reviews Genetics, 10(11), 783–796. doi:10.1038/nrg2664
2. Morton, N. E., Crow, J. F., & Muller, H. J. (1956). "An estimate of the mutational damage in man from data on consanguineous marriages." Proceedings of the National Academy of Sciences, 42(11), 855–863. doi:10.1073/pnas.42.11.855
3. Keller, L. F., & Waller, D. M. (2002). "Inbreeding effects in wild populations." Trends in Ecology & Evolution, 17(5), 230–241. doi:10.1016/S0169-5347(02)02489-802489-8)
4. Hedrick, P. W., & Garcia-Dorado, A. (2016). "Understanding inbreeding depression, purging, and genetic rescue." Trends in Ecology & Evolution, 31(12), 940–952. doi:10.1016/j.tree.2016.09.005