Simple Linear Regression

Simple linear regression models a response variable $y$ as a linear function of one explanatory (predictor) variable $x$, plus random scatter:

Here $\beta_0$ is the true (population) intercept, $\beta_1$ is the true slope, and $\varepsilon_i$ is the deviation of the $i$th individual's actual $y$-value from the line — the part of $y$ not explained by $x$. The Greek letters denote unknown population parameters; biologists estimate them from sample data as $b_0$ and $b_1$, giving the fitted line:

The hat over $y$ signals a predicted value, not an observed one. For any bird with a given femur length $x$, $\hat{y}$ is the regression model's best guess at mean body mass — not a guarantee for that individual bird, but the expected value averaged over many birds with that femur length. Three ideas separate regression from correlation. First, asymmetry: swapping which variable is $x$ and which is $y$ changes the regression line (the line predicting $y$ from $x$ is generally not the same line as the one predicting $x$ from $y$), whereas the correlation coefficient $r$ is unchanged by the swap. Second, units and interpretation: $r$ is a unitless index from $-1$ to $+1$, while the slope $b_1$ carries units (mass per mm, for instance) and has direct biological meaning — a rate of change. Third, purpose: correlation answers "how strongly are these associated?"; regression answers "what value of $y$ do we predict for a given $x$, and by how much does $y$ change per unit of $x$?" Both techniques assume the same underlying bivariate structure, and in fact $b_1$ and $r$ are mathematically linked: $b_1 = r \cdot (s_y / s_x)$, where $s_y$ and $s_x$ are the sample standard deviations of $y$ and $x$. Regression is correlation's more committed, directional cousin.

Step 1 — Specify the model. The biologist chooses which variable is the predictor based on the scientific question, not the data. If the question is "does femur length predict mass," $x$ = femur length and $y$ = mass. In a controlled experiment (e.g., enzyme velocity at set substrate concentrations), the manipulated variable is always $x$.

Step 2 — Assumptions the model requires. For the fitted line to be meaningful and (later) for inference to be valid, several conditions should approximately hold:

• Linearity — the true relationship between the mean of $y$ and $x$ is a straight line.
• Independence — errors ($\varepsilon_i$) for different observations are independent of one another.
• Homoscedasticity — the scatter of $y$ around the line has constant variance across all values of $x$ (no "fanning out").
• Normality of residuals — for a given $x$, the values of $y$ are normally distributed around the line (most critical for inference, less so for merely fitting the line).
• $x$ measured with little error — classical regression assumes the predictor is fixed or measured precisely; heavy measurement error in $x$ biases the slope toward zero.

Step 3 — Fit the line by least squares. Among all possible straight lines, regression picks the one minimizing the sum of squared residuals:

A residual $e_i = y_i - \hat{y}_i$ is the vertical distance between an observed point and the line — the part of that individual's $y$-value the model fails to explain. Squaring penalizes large deviations disproportionately and makes the minimization solvable in closed form (calculus shows the minimizing line has a unique slope and intercept, derived by setting partial derivatives of $SS_{residual}$ with respect to $b_0$ and $b_1$ to zero). This yields:

The slope is the ratio of how $x$ and $y$ co-vary to how much $x$ alone varies — essentially, covariance divided by variance of the predictor. The line necessarily passes through the point $(\bar{x}, \bar{y})$, the centroid of the data — a useful mental anchor.

Step 4 — Partition variance. Total variability in $y$ ($SS_{total} = \sum (y_i - \bar{y})^2$) splits into variability explained by the line ($SS_{regression}$) and variability left over ($SS_{residual}$):

The proportion explained is the coefficient of determination, $r^2$ — identical numerically to the square of the correlation coefficient in simple linear regression. An $r^2$ of 0.81 means 81% of the variance in body mass is statistically accounted for by femur length; the remaining 19% reflects individual variation, measurement error, and other unmeasured biological factors.

Step 5 — Interpret slope and intercept biologically. The slope $b_1$ is the predicted change in mean $y$ per one-unit increase in $x$. The intercept $b_0$ is the predicted $y$ when $x = 0$ — biologically meaningful only if $x = 0$ falls within (or near) the observed range of data; extrapolating the intercept far outside the data (e.g., predicting mass at femur length = 0) is usually nonsensical and a common misuse.

Simple linear regression's method — least squares — was developed independently by Adrien-Marie Legendre (1805) and Carl Friedrich Gauss (who claimed prior use, published 1809), originally to fit orbits of comets and asteroids from astronomical observations. But the biological birth of regression, and its name, comes from Francis Galton. In the 1880s Galton was obsessed with quantifying heredity. Using records from his "anthropometric laboratory," he measured the heights of 205 pairs of parents (averaging mother's and father's height, the latter scaled by a factor of 1.08 to make sexes comparable — Galton's "mid-parent height") and the adult heights of 928 of their offspring. He plotted offspring height against mid-parent height and fitted a line through the data. The result, published in "Regression Towards Mediocrity in Hereditary Stature" (1886), was striking: tall parents tended to have tall children, but on average, less tall than the parents themselves. Children of parents who stood far above the mean tended to be closer to the mean than their parents were. Symmetrically, children of unusually short parents tended to be taller than their parents. Galton calculated that offspring height moved back toward the population mean by a consistent fraction each generation — he called this "regression toward mediocrity" (now known as regression toward the mean).

What this proved: The slope of the offspring-on-midparent line was reliably less than 1 (Galton found approximately 2/3), meaning extreme parental heights predicted less extreme, but still directionally correct, offspring heights. This was not evidence against heredity — tall parents still had taller-than-average children — but it demonstrated that any trait influenced by both heritable and non-heritable (environmental, developmental, chance) components will show this compression toward the mean across generations, a purely statistical consequence of imperfect correlation rather than a biological "regression" of the species. Galton's numerical method for drawing that best-fit line, and his coining of the term "regression" for the fitted slope, became the foundation of the whole technique — and the reason a modeling tool used across all of science is named after a 19th-century height study, not after the mathematics of least squares itself.

A modern echo of Galton's discovery is the "sophomore slump" or regression-to-the-mean artifact seen throughout biology and medicine: patients selected for extremely high blood pressure on one visit tend to show lower readings on a second visit even with no treatment, simply because extreme measurements partly reflect transient noise. Recognizing this is essential to correctly designing and interpreting biological experiments — a direct descendant of what Galton's regression line first revealed. A second classic strand of evidence for the predictive power of regression is Julian Huxley's allometric scaling studies (1932), where he showed that log(organ size) regressed linearly on log(body size) across species and within growth series — for example fiddler crab claw mass against body mass — establishing that simple linear regression on log-transformed variables (allometry) accurately captures growth-scaling laws across biology, from crab claws to mammalian brain size.

Example 1 — Predicting metabolic rate from body mass (allometric regression). A physiologist measures resting metabolic rate ($y$, in mL O₂/hr) and body mass ($x$, in kg) in 8 mammal species spanning three orders of magnitude in size. After log₁₀-transforming both variables (standard practice for allometric data because the relationship is multiplicative, not additive), she obtains:

Slope:

Intercept:

Fitted line: $\log_{10}(\text{metabolic rate}) = 1.497 + 0.711 \times \log_{10}(\text{mass})$. The slope of 0.711 is close to the classic ¾-power (Kleiber's) scaling law for metabolic rate against mass, an empirical regularity across mammals from mice to elephants. To predict metabolic rate for a new species weighing 25 kg:

$r^2 = 0.93^2 = 0.865$: about 86.5% of the variance in log metabolic rate across these species is explained by log body mass, leaving 13.5% attributable to metabolic differences between taxa (e.g., basal metabolic differences between marsupials and placental mammals) not captured by mass alone.

Example 2 — Dose–response calibration curve in a toxicology assay. A toxicologist exposes fish larvae to five concentrations of a pesticide ($x$, µg/L) and records percentage mortality after 96 hours ($y$, %), replicated with asymmetric spacing typical of real dose-ranging studies:

(For teaching purposes we treat the mid-range of this dose–response relationship as approximately linear; true dose–response curves are sigmoidal and normally require probit or logistic regression — a simplification flagged deliberately here.) Computing the sums:

Fitted line: $\hat{y} = 17.50 + 2.834x$.

Biological interpretation: each additional 1 µg/L of pesticide is associated with an estimated 2.83-percentage-point increase in larval mortality across this dose range. The intercept, 17.5%, would represent predicted mortality at zero dose — clearly higher than the observed near-zero mortality at the lowest tested dose, a signal that the true relationship curves rather than continuing linearly down to $x=0$, and a caution against extrapolating below the tested range. Using the line to interpolate the dose producing 50% mortality (an approximate LD50):

This falls just outside the tested range (max tested = 10 µg/L), illustrating both the predictive utility of regression and the risk of extrapolation beyond observed data — the LD50 estimate should be treated cautiously and ideally confirmed with an additional trial closer to that dose.

Clinical medicine: regression models predict drug dosage from body weight or renal function (creatinine clearance), estimate cardiovascular risk from cholesterol and blood pressure, and calibrate assay readings (e.g., converting spectrophotometer absorbance to hormone concentration) via standard curves fit by least squares.

Growth and development: pediatric growth charts and animal husbandry weight-gain curves use regression of mass or height against age to flag individuals deviating from expected trajectories.

Ecology and conservation: regression of species richness against habitat area (species–area curves), population growth rate against resource availability, or contaminant concentration against distance from a pollution source, all use the same least-squares logic to make quantitative field predictions.

Agriculture: yield response to fertilizer dose, feed conversion ratio against animal weight, and calibration of remote-sensing vegetation indices against actual crop biomass all rely on fitted regression lines (or their generalizations) to guide practical decisions.

Genomics and biotechnology: standard curves in qPCR (cycle threshold against known template concentration) and ELISA assays are simple linear regressions (often on log-transformed concentration) used daily in molecular biology labs to quantify unknown samples.

Evolutionary biology: regression remains central to quantitative genetics — the modern descendant of Galton's own parent–offspring height study, where the regression slope of offspring trait value on mid-parent value estimates the heritability ($h^2$) of a trait, connecting this 19th-century method directly to present-day breeding programs and evolutionary predictions.

Ask the tutor: How does adding a second predictor variable turn simple linear regression into multiple regression, and what new problems (like collinearity) does that introduce?