Variance and standard deviation

Two data sets can share the same mean and yet feel completely different — one tightly bunched, the other scattered wide. Range only looks at the two extremes, and mean deviation dodges the sign problem by taking absolute values, which is awkward to manipulate algebraically. Variance and standard deviation solve this more elegantly: square every deviation from the mean instead of taking its absolute value. Squaring kills the sign just as effectively, but the result behaves beautifully under algebra — sums of squares expand, factor and combine in ways absolute values never do, which is exactly why every later statistical idea (correlation, regression, the normal distribution) is built on variance rather than mean deviation. This section covers the definitions for ungrouped and grouped data, and the three computational routes — direct, short-cut (assumed-mean) and step-deviation — that all arrive at the same answer.

The core insight

Variance is the mean of the squared deviations from the mean, σ2=∑fi(xi−xˉ)2∑fi\sigma^2 = \dfrac{\sum f_i(x_i-\bar x)^2}{\sum f_i}, and standard deviation σ=σ2\sigma = \sqrt{\sigma^2} restores the original units; the short-cut and step-deviation methods compute the same σ\sigma with smaller numbers.

The core idea

Why squares, not absolute values. Mean deviation about the mean, 1n∑∣xi−xˉ∣\frac{1}{n}\sum|x_i-\bar x|, avoids the deviations cancelling to zero, but the modulus sign makes it hard to differentiate or combine algebraically. Squaring achieves the same cancellation-proofing while staying smooth and additive — this is the whole reason variance, not mean deviation, is the measure carried forward into every advanced statistical technique.

Definitions. For grouped data with values xix_i occurring with frequency fif_i and mean xˉ\bar x, variance is the average squared deviation:

σ2=∑fi(xi−xˉ)2∑fi\sigma^2 = \frac{\sum f_i(x_i - \bar x)^2}{\sum f_i}

where: xix_i are the data values (or class mid-points), fif_i are the corresponding frequencies, xˉ\bar x is the mean, and ∑fi=n\sum f_i = n is the total number of observations. For ungrouped data every fi=1f_i = 1, giving σ2=∑(xi−xˉ)2n\sigma^2 = \dfrac{\sum(x_i-\bar x)^2}{n}. Standard deviation is the positive square root — it brings the "squared units" of variance (e.g. cm² for a height data set) back to the original units (cm), which is why it, not variance, is usually the number reported:

σ=∑fi(xi−xˉ)2∑fi\sigma = \sqrt{\frac{\sum f_i(x_i-\bar x)^2}{\sum f_i}}

Three routes to the same number. The direct method applies the definition above as-is — fine when xˉ\bar x and the xix_i are small, tedious otherwise. The short-cut (assumed-mean) method picks a convenient value AA near the centre of the data, sets di=xi−Ad_i = x_i - A, and uses

σ=∑fidi2∑fi−(∑fidi∑fi)2\sigma = \sqrt{\frac{\sum f_i d_i^2}{\sum f_i} - \left(\frac{\sum f_i d_i}{\sum f_i}\right)^2}

which is algebraically identical to the direct formula but keeps the numbers small. The step-deviation method, for grouped data with equal class width hh, further scales ui=xi−Ahu_i = \dfrac{x_i - A}{h} so the uiu_i become small integers:

σ=h∑fiui2∑fi−(∑fiui∑fi)2\sigma = h\sqrt{\frac{\sum f_i u_i^2}{\sum f_i} - \left(\frac{\sum f_i u_i}{\sum f_i}\right)^2}

where: AA is any assumed mean (a convenient reference value, usually a class mid-point), di=xi−Ad_i = x_i - A, hh is the common class width, and ui=di/hu_i = d_i/h. All three formulas give the identical σ\sigma; only the arithmetic load differs.

Figure
Three data clusters with the same mean but growing spread, tall narrow bars for low variance and short wide bars for high variance
Same mean, growing spread — variance measures the difference mean low sigma medium sigma high sigma

Worked example

The heights (in cm) of 50 plants are grouped as below. Find the standard deviation using the step-deviation method, with assumed mean A=35A = 35 and class width h=10h = 10.

Classfif_ixix_iui=xi−3510u_i = \frac{x_i-35}{10}fiuif_iu_ifiui2f_iu_i^2
10–20515−2-2−10-1020
20–301225−1-1−12-1212
30–40183500000
40–50104511101010
50–6055522101020
Total50−2-262

Step 1. Read off the totals: ∑fi=50\sum f_i = 50, ∑fiui=−2\sum f_iu_i = -2, ∑fiui2=62\sum f_iu_i^2 = 62.

Step 2. Apply the step-deviation formula: σ=h∑fiui2∑fi−(∑fiui∑fi)2=106250−(−250)2\sigma = h\sqrt{\dfrac{\sum f_iu_i^2}{\sum f_i} - \left(\dfrac{\sum f_iu_i}{\sum f_i}\right)^2} = 10\sqrt{\dfrac{62}{50} - \left(\dfrac{-2}{50}\right)^2}.

Step 3. Compute inside the root: 6250=1.24\dfrac{62}{50} = 1.24 and (−250)2=0.0016\left(\dfrac{-2}{50}\right)^2 = 0.0016, so σ=101.24−0.0016=101.2384≈10×1.1128\sigma = 10\sqrt{1.24 - 0.0016} = 10\sqrt{1.2384} \approx 10 \times 1.1128.

Step 4. The mean can be recovered from the same table as a check: xˉ=A+h⋅∑fiui∑fi=35+10×−250=34.6\bar x = A + h\cdot\dfrac{\sum f_iu_i}{\sum f_i} = 35 + 10\times\dfrac{-2}{50} = 34.6 cm, consistent with the assumed mean A=35A = 35 being close to centre. Standard deviation σ≈11.13\sigma \approx 11.13 cm, so variance σ2≈123.9\sigma^2 \approx 123.9 cm².

Common traps

  • Reporting variance (σ2\sigma^2) when the question asks for standard deviation (σ\sigma), or vice versa — always check the units expected in the answer.
  • Forgetting to multiply by hh at the end of the step-deviation formula, which silently gives a value that is too small.
  • Squaring the whole fraction ∑fiui∑fi\dfrac{\sum f_iu_i}{\sum f_i} correctly, but then subtracting (∑fiui)2∑fi\dfrac{(\sum f_iu_i)^2}{\sum f_i} instead — these two expressions are not the same and give a different, wrong result.
  • Choosing an assumed mean AA far from the data's centre, which makes did_i or uiu_i unnecessarily large and defeats the point of the short-cut and step-deviation methods.
Derivation AThe short-cut formula gives the same variance as the direct formula

Show algebraically that switching from the direct deviations xi−xˉx_i - \bar x to the assumed-mean deviations di=xi−Ad_i = x_i - A produces an identical value of σ2\sigma^2, so the choice of AA never changes the answer — only the size of the numbers being handled changes, which is the entire justification for using a shortcut at all.

Step 1. Write di=xi−Ad_i = x_i - A, so xi=A+dix_i = A + d_i. The mean satisfies xˉ=A+dˉ\bar x = A + \bar d, where dˉ=∑fidi∑fi\bar d = \dfrac{\sum f_id_i}{\sum f_i}, because averaging both sides of xi=A+dix_i = A + d_i over the frequencies just adds AA to the average of did_i.

Step 2. Substitute into the deviation from the mean: xi−xˉ=(A+di)−(A+dˉ)=di−dˉx_i - \bar x = (A + d_i) - (A + \bar d) = d_i - \bar d. So the deviation of xix_i from xˉ\bar x equals the deviation of did_i from dˉ\bar d — shifting every value by the constant AA does not change how spread out they are.

Step 3. Square and average: σ2=∑fi(xi−xˉ)2∑fi=∑fi(di−dˉ)2∑fi\sigma^2 = \dfrac{\sum f_i(x_i-\bar x)^2}{\sum f_i} = \dfrac{\sum f_i(d_i-\bar d)^2}{\sum f_i}. Expand the right-hand side: ∑fi(di2−2didˉ+dˉ2)∑fi=∑fidi2∑fi−2dˉ⋅∑fidi∑fi+dˉ2\dfrac{\sum f_i(d_i^2 - 2d_i\bar d + \bar d^2)}{\sum f_i} = \dfrac{\sum f_id_i^2}{\sum f_i} - 2\bar d\cdot\dfrac{\sum f_id_i}{\sum f_i} + \bar d^2.

Step 4. Since dˉ=∑fidi∑fi\bar d = \dfrac{\sum f_id_i}{\sum f_i}, the middle term is −2dˉ2-2\bar d^2, so the expression collapses to ∑fidi2∑fi−2dˉ2+dˉ2=∑fidi2∑fi−dˉ2=∑fidi2∑fi−(∑fidi∑fi)2\dfrac{\sum f_id_i^2}{\sum f_i} - 2\bar d^2 + \bar d^2 = \dfrac{\sum f_id_i^2}{\sum f_i} - \bar d^2 = \dfrac{\sum f_id_i^2}{\sum f_i} - \left(\dfrac{\sum f_id_i}{\sum f_i}\right)^2, which is exactly the short-cut formula. Since AA never appears in the final expression except inside did_i, any choice of AA gives the same σ2\sigma^2 — only the arithmetic effort changes.

Ask the tutor: if your step-deviation answer does not match the direct-method answer, or you want a fresh grouped data set to practise all three methods on, ask the AI tutor to walk through it with you.

Why do we square the deviations instead of just adding the negative ones as positive?

Adding absolute values, ∑∣xi−xˉ∣\sum|x_i - \bar x|, does avoid the deviations cancelling out, and that is exactly what mean deviation does. But absolute value is a "kinked" function — it has no clean derivative at zero and does not expand algebraically the way a square does. Squaring keeps everything smooth: (xi−xˉ)2(x_i-\bar x)^2 can be expanded, summed, and combined with other squared quantities using ordinary algebra, which is precisely why variance underlies correlation, regression and the normal distribution later on, while mean deviation stays a comparatively minor measure.

Does it matter which value I choose for the assumed mean A?

Not for correctness — any value of AA gives exactly the same final standard deviation, as the derivation above shows algebraically. It matters only for convenience: picking AA far from the data's centre makes the did_i or uiu_i values large, which brings back the very arithmetic burden the short-cut method was meant to avoid. The usual choice is a value near the middle of the data, often the mid-point of the class with the highest frequency.

If I already have variance, how do I know when to also report standard deviation?

Report standard deviation whenever the answer needs to be in the same units as the original data — variance carries squared units (cm² for heights, marks² for scores), which are not directly interpretable, while standard deviation is back in cm or marks and can be compared directly to the data itself. Many exam questions ask for both: compute variance first as σ2\sigma^2, then take its square root for σ\sigma, keeping enough decimal places before rounding so the final square root is accurate.

Before this

Where this leads