Nonlinear Relationships
When we collect data and try to model the relationship between two variables, we often start by fitting a straight line — a linear regression. But nature, economics, biology, and physics frequently produce relationships that curve, accelerate, or level off. A nonlinear relationship is one where the rate of change between the explanatory variable x and the response variable y is not constant; the scatter plot bends rather than following a straight path. Recognising these patterns before choosing a model is one of the most important skills in data exploration. Fitting a linear model to curved data is a classic example of model inadequacy — the model is technically computable, but it misrepresents the underlying process. The consequences can be severe: poor predictions, misleading conclusions, and residuals (the differences between observed and predicted values) that carry hidden structure. Two primary tools help us detect nonlinearity: visual inspection of the original scatter plot, and analysis of the residual plot after fitting a linear model. Once nonlinearity is confirmed, statisticians transform variables or choose alternative models — exponential, power, logarithmic, or quadratic — to better capture the true relationship. This article explains how to detect nonlinear patterns, interpret residual plots, and understand why choosing the right model matters for valid statistical inference.
- Examine the Scatter Plot First
- Fit a Linear Model and Examine Residuals
- Linearise Using Transformations
- Assess Model Adequacy with r² and Residual Patterns
- Choose and Interpret the Nonlinear Model
Examine the Scatter Plot First
Always begin by plotting y against x. A linear relationship shows points scattered roughly around an imaginary straight line with no systematic curve. A nonlinear relationship shows points that arch upward, curve downward, form a U-shape, or follow an S-shape. Common patterns include exponential growth (points curve sharply upward), logarithmic growth (points rise quickly then flatten), and quadratic relationships (a symmetric arch). If the scatter plot curves, a linear model is inadequate before you even compute anything.
Fit a Linear Model and Examine Residuals
Fit the least-squares regression line ŷ = a + bx and compute each residual: eᵢ = yᵢ − ŷᵢ. Plot these residuals on the vertical axis against the x-values (or fitted values ŷ) on the horizontal axis. If the linear model is appropriate, residuals should scatter randomly around zero with no pattern — like horizontal noise. If you see a curved pattern (e.g., residuals that are negative, then positive, then negative again as x increases), this is strong evidence that the linear model is inadequate and a nonlinear model is needed.
Linearise Using Transformations
Many nonlinear relationships can be converted ('linearised') into linear ones by transforming variables. For exponential data where y = ab^x, taking the logarithm of both sides gives log(y) = log(a) + x·log(b), which is linear in x and log(y). For power relationships y = ax^n, taking log of both sides gives log(y) = log(a) + n·log(x), linear in log(x) and log(y). After transformation, fit a linear regression to the transformed variables, check that the new residual plot looks random, then interpret predictions by reversing the transformation.
Assess Model Adequacy with r² and Residual Patterns
The coefficient of determination r² measures what fraction of variability in y is explained by the model. A high r² from a linear fit does not guarantee the model is appropriate — you must check the residual plot. Conversely, a low r² after transformation might suggest neither model fits well. A well-fitting nonlinear model will have: (1) a random, patternless residual plot, (2) residuals with roughly constant spread (homoscedasticity), and (3) an r² value meaningful in context. Always report both r² and a description of the residual plot when justifying a model choice.
Choose and Interpret the Nonlinear Model
Once you identify the appropriate transformation, write the model in its original (untransformed) form for interpretation. For an exponential model, if log(y) = 0.301 + 0.150x, then y = 10^(0.301) · 10^(0.150x) ≈ 2.00 · (1.413)^x. This means y approximately doubles from a starting value of 2, multiplying by 1.413 for each unit increase in x. Always convert back to the original scale and interpret the model's parameters in context — growth rate, initial value, curvature — to make the results meaningful.
The Key Equation
- \log(y) — Transformed response variable (dimensionless): The common logarithm (base 10) of the original response variable y; this transformation straightens exponential curves.
- \log(a) — y-intercept of transformed model (dimensionless): The vertical intercept of the linearised model; reversing gives a = 10^(log a), the initial value of y when x = 0.
- b — Slope of linearised model (per unit of x): The slope in the log-transformed model; in the original exponential model, 10^b gives the multiplicative factor by which y changes for each unit increase in x.
- x — Explanatory variable (varies by context): The independent variable, which enters the linearised model without transformation in the exponential case.
Worked Example
Problem. A biologist records the number of bacteria (y, in thousands) in a culture at time x (hours): (0, 2), (1, 4), (2, 8), (3, 16), (4, 32). She fits a linear regression and gets ŷ = −6 + 8x with r² = 0.90. The residuals at x = 0,1,2,3,4 are 8, 2, −2, −2, 8. Is the linear model adequate? Suggest and fit a better model.
Answer. The linear model is inadequate because residuals show a curved (U-shaped) pattern. The correct model is exponential: y = 2 · 2^x, or equivalently log(y) = 0.301 + 0.301x.
Origins · 1 minFrancis Galton and how this was discovered
Francis Galton pioneered the mathematical study of regression and correlation in the 1880s. While studying the relationship between heights of parents and children, he observed that children of very tall parents tended to be shorter than their parents — a phenomenon he called 'regression toward mediocrity' (now called regression to the mean). His work laid the quantitative foundation for fitting models to data and measuring association between variables, directly enabling the study of both linear and nonlinear relationships. Galton invented the weather map and was the first to recognise the uniqueness of fingerprints for identification — he estimated the probability of two people sharing the same fingerprint was 1 in 64 billion.
Related Concepts
- Linear Regression — Nonlinear relationship detection begins by fitting and then diagnosing the inadequacy of a linear regression model; understanding linear regression is the essential prerequisite.
- Residual Analysis — Residual plots are the primary diagnostic tool for detecting nonlinearity and model inadequacy; residual analysis is inseparable from evaluating any regression model.
- Variable Transformations in Regression — Logarithmic and power transformations linearise nonlinear data, making them the key technique for fitting appropriate models once nonlinearity is identified.
- Coefficient of Determination (r²) — r² is used alongside residual plots to assess model fit; understanding its limitations — it can be high even for inadequate models — is critical in nonlinear contexts.