Bivariate Data

Bivariate data consists of paired observations of two variables measured on the same individuals, used to investigate whether and how the two variables are related. Key result in words: the strength and direction of a linear relationship between two numerical variables is captured by the correlation coefficient

r

, and the relationship can be modelled by a least-squares regression line. Applies wherever you want to study association — height vs. weight, study time vs. exam score, temperature vs. ice-cream sales. Developed mainly by Francis Galton and Karl Pearson in the late 19th century. Bivariate data is data in which each unit of observation carries two values, one for each of two variables — written as ordered pairs

(x_{i}, y_{i})

. Whereas univariate statistics summarise a single variable (its centre, spread, shape), bivariate statistics ask the new question: do the two variables move together? This matters because almost every interesting scientific or commercial question is about a relationship — does a drug lower blood pressure, does advertising raise sales, does smoking shorten life? A surprising fact is that two variables can be perfectly related in a curved way (e.g.

y = x^{2}

over a symmetric range) yet have a correlation coefficient of exactly zero — correlation only measures the linear part of a relationship.

The Core Insight

A single pair of numbers tells you nothing about a relationship; a cloud of pairs does. By plotting all pairs as points (a scatterplot) you can see a relationship, and by computing how the two variables co-vary about their means you can measure it. The central quantity is covariance — the average of the products of paired deviations from the means — which, once you divide out the two standard deviations, becomes the dimensionless, scale-free correlation coefficient between $- 1$ and $+ 1$ .

Formal Statement

Bivariate Data and Its Building Blocks

A bivariate data set of size $n$ is a list of ordered pairs

{(x_{1}, y_{1}), (x_{2}, y_{2}), \dots, (x_{n}, y_{n})} .

⟳this is a definition — each individual contributes one

x

-value and one

y

-value measured simultaneously.

We summarise each variable by its mean and standard deviation:

\overset{x}{ˉ} = \frac{1}{n} i = 1 \sum n x_{i}, \overset{y}{ˉ} = \frac{1}{n} i = 1 \sum n y_{i} .

⟳ordinary arithmetic averages applied separately to the

x

-list and the

y

-list.

The sample covariance measures how the two vary together:

s_{x y} = \frac{1}{n - 1} i = 1 \sum n (x_{i} - \overset{x}{ˉ}) (y_{i} - \overset{y}{ˉ}) .

⟳average the product of each point's

x

-deviation and

y

-deviation; positive products (both above or both below their means) push covariance up.

The (Pearson) correlation coefficient is the standardised covariance:

r = \frac{s _{x y}}{s _{x} s _{y}} = \frac{\sum _{i = 1}^{n} ( x _{i} - x ˉ ) ( y _{i} - y ˉ )}{\sum _{i = 1}^{n} ( x _{i} - x ˉ ) ^{2} \sum _{i = 1}^{n} ( y _{i} - y ˉ ) ^{2}} .

⟳divide covariance by the product of the two standard deviations; the

(n - 1)

factors cancel between numerator and denominator, giving a unitless number.

Here $s_{x}, s_{y}$ are the standard deviations of $x$ and $y$ , and the symbols are: $n$ = number of pairs, $x_{i}, y_{i}$ = the values of the two variables for individual $i$ , $\overset{x}{ˉ}, \overset{y}{ˉ}$ = means, $s_{x y}$ = covariance, $r$ = correlation coefficient.

Properties of the Correlation Coefficient

For any bivariate data set:

- 1 \leq r \leq + 1.

⟳follows from the Cauchy–Schwarz inequality applied to the deviation vectors

u_{i} = x_{i} - \overset{x}{ˉ}

and

v_{i} = y_{i} - \overset{y}{ˉ}

Furthermore $r = + 1$ (resp. $r = - 1$ ) exactly when the points lie on a straight line of positive (resp. negative) slope, and $r$ is unchanged by any linear rescaling $x \mapsto a x + b, y \mapsto cy + d$ with $a, c > 0$ .

Least-Squares Regression Line

The line $\overset{y}{^} = a + b x$ minimising the sum of squared vertical residuals $\sum (y_{i} - \overset{y}{^}_{i})^{2}$ has

b = \frac{s _{x y}}{s _{x}^{2}} = r \frac{s _{y}}{s _{x}}, a = \overset{y}{ˉ} - b \overset{x}{ˉ} .

⟳set the partial derivatives of

\sum (y_{i} - a - b x_{i})^{2}

with respect to

a

and

b

to zero and solve the resulting two "normal equations."

The proportion of variation in $y$ explained by the line is the coefficient of determination $r^{2}$ .

Why It Holds / Derivation

Deriving the Least-Squares Slope and Intercept

We minimise the total squared error as a function of intercept $a$ and slope $b$ :

Define $S (a, b) = \sum_{i = 1}^{n} (y_{i} - a - b x_{i})^{2}$ .

Differentiate with respect to $a$ and set to zero:

\frac{\partial S}{\partial a} = - 2 i = 1 \sum n (y_{i} - a - b x_{i}) = 0.

⟳chain rule on each squared term; the constant

- 2

factors out.

Dividing by $- 2$ and rearranging gives $\sum y_{i} = na + b \sum x_{i}$ , i.e. $\overset{y}{ˉ} = a + b \overset{x}{ˉ}$ , so $a = \overset{y}{ˉ} - b \overset{x}{ˉ}$ . The line passes through $(\overset{x}{ˉ}, \overset{y}{ˉ})$ .

Differentiate with respect to $b$ and set to zero:

\frac{\partial S}{\partial b} = - 2 i = 1 \sum n x_{i} (y_{i} - a - b x_{i}) = 0.

⟳same chain rule, but the inner derivative with respect to

b

brings down a factor

x_{i}

Substitute $a = \overset{y}{ˉ} - b \overset{x}{ˉ}$ into step 4. After expanding and cancelling, the cross terms collapse to the covariance and variance:

b = \frac{\sum ( x _{i} - x ˉ ) ( y _{i} - y ˉ )}{\sum ( x _{i} - x ˉ ) ^{2}} = \frac{s _{x y}}{s _{x}^{2}} .

⟳use

\sum x_{i} (y_{i} - \overset{y}{ˉ}) = \sum (x_{i} - \overset{x}{ˉ}) (y_{i} - \overset{y}{ˉ})

, valid because

\sum (y_{i} - \overset{y}{ˉ}) = 0

Since $s_{x y} = r s_{x} s_{y}$ , we get $b = r s_{y} / s_{x}$ , completing equation [6]. $■$

Karl Pearson | 1857–1936

Pearson, a polymath at University College London, transformed the loose Victorian study of "biometry" into rigorous mathematical statistics. Building on Francis Galton's idea of regression toward mediocrity, in 1896 he gave the product-moment correlation coefficient its modern algebraic form — equation [4] still bears the name Pearson's $r$ . He founded the world's first university statistics department and the journal Biometrika. His work, though scientifically foundational, was entangled with the eugenics movement of his era, a connection modern readers should view critically.

Building Intuition

Picture each data point's contribution to the covariance sum [3]. Split the scatterplot into four quadrants by drawing vertical and horizontal lines through the point of means $(\overset{x}{ˉ}, \overset{y}{ˉ})$ . A point in the upper-right quadrant has both deviations positive, so its product is positive; a point in the lower-left has two negatives, also a positive product. Points in those two "agreeing" quadrants vote for a positive relationship. Points in the upper-left and lower-right quadrants have one positive and one negative deviation, contributing negative products and voting for a negative relationship. The covariance is the net tally of these votes. Dividing by $s_{x} s_{y}$ just converts the tally to a fixed scale from $- 1$ to $+ 1$ , so a tall thin person and a short fat person produce the same $r$ as the same shapes measured in inches versus centimetres.

Scatterplot with least-squares line and the four deviation quadrants about the point of means

Worked Examples

Correlation between study hours and test score

Given five students' weekly study hours

x

and test score

y

| $x$ | 2 | 3 | 5 | 7 | 8 | |---|---|---|---|---|---| | $y$ | 50 | 60 | 70 | 80 | 90 |

Step 1 — means.

\overset{x}{ˉ} = (2 + 3 + 5 + 7 + 8) /5 = 5

\overset{y}{ˉ} = (50 + 60 + 70 + 80 + 90) /5 = 70

. Step 2 — deviations and products.

| $x_{i} - \overset{x}{ˉ}$ | $y_{i} - \overset{y}{ˉ}$ | product | $(x_{i} - \overset{x}{ˉ})^{2}$ | $(y_{i} - \overset{y}{ˉ})^{2}$ | |---|---|---|---|---| | $- 3$ | $- 20$ | $60$ | $9$ | $400$ | | $- 2$ | $- 10$ | $20$ | $4$ | $100$ | | $0$ | $0$ | $0$ | $0$ | $0$ | | $2$ | $10$ | $20$ | $4$ | $100$ | | $3$ | $20$ | $60$ | $9$ | $400$ |

Sums: $\sum (x - \overset{x}{ˉ}) (y - \overset{y}{ˉ}) = 160$ , $\sum (x - \overset{x}{ˉ})^{2} = 26$ , $\sum (y - \overset{y}{ˉ})^{2} = 1000$ .

Step 3 — correlation (equation [4]).

r = \frac{160}{26 1000} = \frac{160}{161.2} = 0.992.

Step 4 — regression slope and intercept (equation [6]).

b = \frac{160}{26} = 6.15, a = 70 - 6.15 (5) = 39.2.

r = 0.99 (strong positive), \overset{y}{^} = 39.2 + 6.15 x (score per study-hour)

Predicting with the regression line

Given the line

\overset{y}{^} = 39.2 + 6.15 x

from the previous example, predict the score of a student who studies 6 hours, and report how much of the variation in scores the model explains. Step 1 — substitute $x = 6$ .

\overset{y}{^} = 39.2 + 6.15 (6) = 39.2 + 36.9 = 76.1.

Step 2 — coefficient of determination.

r^{2} = (0.992)^{2} = 0.984

. Interpretation. About 98.4 % of the variation in test scores is accounted for by study hours under this linear model; the predicted score for 6 hours is roughly 76 points. Note

x = 6

lies inside the data range

[2, 8]

, so this is legitimate interpolation, not risky extrapolation.

\overset{y}{^} \approx 76 points, r^{2} = 0.98

Common Misconceptions

Correlation is not causation

A strong $r$ tells you two variables move together; it does not tell you that one causes the other. The relationship may be reversed, coincidental, or driven by a hidden confounding variable. Classic example: across towns, the number of churches correlates strongly with the number of bars — but neither causes the other; both are driven by population size. Only a controlled experiment or careful causal reasoning can establish causation.

Do not extrapolate beyond the data

The regression line is only validated within the range of observed $x$ -values. Plugging in an $x$ far outside that range (extrapolation) can give absurd answers — e.g. extending a child's height-vs-age line to age 40 predicts a 5-metre adult. Always check that your prediction's $x$ lies inside the data spread.

Zero correlation despite a perfect relationship

Take the symmetric data $(- 2, 4), (- 1, 1), (0, 0), (1, 1), (2, 4)$ , which lies exactly on the parabola $y = x^{2}$ . Here $\overset{x}{ˉ} = 0$ and the products $(x_{i} - \overset{x}{ˉ}) (y_{i} - \overset{y}{ˉ})$ are $(- 2) (1.8), (- 1) (- 1.2), (0) (- 2.2), (1) (- 1.2), (2) (1.8)$ , which sum to $- 3.6 - (- 1.2) + 0 + (- 1.2) + 3.6 = 0$ . Therefore $r = 0$ . The variables are perfectly (deterministically) related, yet Pearson's $r$ reports "no linear relationship" — proving $r$ measures only the linear component of association. Always look at the scatterplot, never trust $r$ alone.

Connections

Bivariate analysis is the bridge between Univariate Data (single-variable summaries like mean and standard deviation, which feed directly into equations [2]–[4]) and Multivariate Statistics (three or more variables, partial correlation, multiple regression). The correlation coefficient is the sample estimate of the theoretical correlation studied in Probability Theory via the bivariate normal distribution. The least-squares method is one instance of Optimization / Calculus (setting derivatives to zero), and the algebra of deviation vectors links to Inner Product Spaces, where $r$ is literally the cosine of the angle between the centred data vectors.

Real-World Applications

Medicine: correlating dosage with blood-pressure reduction in clinical trials; the Framingham Heart Study used regression to link cholesterol to heart-disease risk.
Economics: the Phillips curve relates unemployment and inflation; demand curves regress quantity sold against price.
Education (PISA, SAT): correlating socioeconomic indicators with test performance to identify achievement gaps.
Sports analytics ("Moneyball"): the Oakland Athletics regressed on-base percentage against runs scored to find undervalued players.
Meteorology and retail: correlating daily temperature with energy demand or ice-cream sales for stock planning.
Genetics and agriculture: Galton's original work correlated parents' and offspring's heights; modern breeders regress crop yield on rainfall and fertiliser.
Astronomy: the Hertzsprung–Russell diagram is a bivariate scatterplot of stellar luminosity against temperature, revealing the main sequence.

Historical Development

1846: Auguste Bravais writes mathematical formulae for the joint distribution of errors, anticipating the correlation concept without naming it.
1877: Francis Galton, studying sweet-pea seeds, observes that offspring sizes "regress toward the mean" — coining regression.
1885: Galton presents the first regression line and correlation table for parent–child stature data, introducing the symbol $r$ for "reversion/regression."
1888: Galton publishes Co-relations and their Measurement, defining correlation as a measurable quantity.
1896: Karl Pearson derives the modern product-moment correlation coefficient (equation [4]) and least-squares regression in rigorous algebraic form.
1900: Pearson introduces the chi-square test, extending association methods to categorical bivariate data.
1904: Charles Spearman invents the rank correlation coefficient for ordinal/monotonic data, sidestepping the linearity assumption.
1908–1925: William Gosset ("Student") and R. A. Fisher develop the sampling distribution of $r$ and tests of its significance, founding inferential regression.
1930s onward: Fisher's Statistical Methods for Research Workers standardises regression and correlation across the experimental sciences.

Subtleties & Limits

Beyond linear, beyond Pearson

Pearson's $r$ captures only linear association and is sensitive to outliers and to the assumption of an underlying bivariate-normal distribution. Several deeper questions remain practically important: (i) How best to quantify general dependence — modern measures such as distance correlation (Székely, 2007) and maximal information coefficient (MIC, 2011) aim to detect any relationship, but no single measure is universally agreed upon. (ii) The problem of distinguishing genuine causation from correlation in observational data is a live research frontier (Pearl's causal calculus, instrumental variables, randomised trials). (iii) When data are heavy-tailed or contaminated, robust regression methods (least absolute deviations, Theil–Sen estimator) are preferred, and choosing among them remains context-dependent. At the introductory level the cautionary lesson is simpler: always plot the data, beware outliers, and never confuse a tidy $r$ with a true model of reality.