Two classes both average 75 on a test. In one, everyone scored in the 70s. In the other, scores ran from 50 to 100. Same mean, completely different reality. Dispersion is what the average leaves out: how much the data varies.
A self-driving car that brakes "on average" at the right moment but with huge variability is dangerous. In finance, spread is risk. Reporting a mean without a measure of spread tells only half the story.
Range, Quartiles & the IQR
The simplest measure is the range (max − min), but it depends entirely on the two most extreme values, so a single outlier wrecks it. The interquartile range (IQR) is sturdier: it's the spread of the middle half of the data.
Sort the data and split it into quarters. Q1 is the 25th percentile, Q3 the 75th, and IQR = Q3 − Q1. The five-number summary (min, Q1, median, Q3, max) is exactly what a box plot draws:
A common, automatic test: any value below Q1 − 1.5×IQR or above Q3 + 1.5×IQR is flagged as a potential outlier. It's exactly how box plots decide which points to draw separately.
Variance & Standard Deviation
These are the workhorses of spread. The idea: measure how far each value is from the mean (its deviation), then average those distances. Two wrinkles make it work.
Variance: square the deviations
Deviations sum to zero (positives cancel negatives), so we square them first. Squaring also makes big misses count more. The catch: the result is in squared units.
Standard deviation: take the root
The square root of the variance brings the number back into the data's own units, which is why we usually report the standard deviation, not the variance.
For a whole population you divide the squared deviations by N (giving σ², σ). For a sample you divide by n − 1 (giving s², s). The reason: the sample mean sits at the center of its own data, so deviations from it run slightly too small. Dividing by the smaller n − 1 (called Bessel's correction) nudges the estimate up to undo that bias. With large n the difference is tiny; it matters most for small samples.
Heads-up on units: if data is in dollars, the variance is in "dollars squared" (not meaningful to read), while the standard deviation is back in dollars. Always report spread with the standard deviation.
Reading the SD: the 68-95-99.7 Rule
A standard deviation only feels meaningful once you can picture it. For data that is roughly bell-shaped (normal), the empirical rule says:
±1 SD ≈ 68%
About two-thirds of values fall within one standard deviation of the mean.
±2 SD ≈ 95%
The vast majority fall within two. Values beyond ±2 SD start to look unusual.
±3 SD ≈ 99.7%
Almost everything. A value past ±3 SD is genuinely rare.
Chebyshev's rule works for any shape of data, not just bell curves: at least 75% of values lie within 2 SD and at least 89% within 3 SD (looser, but always true). And the SD is the unit behind the z-score, z = (value − mean) / SD, which says how many standard deviations from the mean a value is. More on that in the standardization chapter.
Two More Tools: MAD & the Coefficient of Variation
| Measure | What it is | Good for | Watch out |
|---|---|---|---|
| Range | Max − Min | A 5-second feel for spread | Ruined by one outlier |
| IQR | Q3 − Q1 (middle 50%) | Skewed data; box plots | Ignores the tails entirely |
| Std deviation | √(avg squared deviation) | Bell-shaped data; most of statistics | Sensitive to outliers |
| MAD | Avg of |value − mean| | An intuitive, robust alternative to SD | Less tidy mathematically |
| CV | SD ÷ mean (often ×100%) | Comparing spread across different units/scales | Needs positive data with a true zero |
Is height (in cm) more variable than weight (in kg)? You can't compare their standard deviations directly, the units differ. The CV = SD / mean is unitless, so it can: a higher CV means more relative spread. Just don't use it when the mean is near zero or the data can be negative.
Use mean ± standard deviation for roughly symmetric data with no wild outliers. Use the median + IQR (and a box plot) for skewed or outlier-prone data, because, like the median, the IQR shrugs off extreme values.
Dispersion in Machine Learning & AI
Spread is not just a summary statistic, it is a quantity models are built on and tuned against. The word variance even names one half of the central trade-off in all of supervised learning.
| Idea | Where it shows up in ML | Why |
|---|---|---|
| Standard deviation | The denominator in standardization (z-scores), which most models need | Dividing by the SD puts every feature on a comparable scale so none dominates by unit alone |
| Variance | The bias-variance trade-off; near-zero-variance features are dropped in preprocessing | A feature that barely varies carries almost no signal; a high-variance model overfits noise |
| Coefficient of variation | Comparing the noisiness of features or metrics measured on different scales | Relative spread lets you compare a millisecond latency to a dollar amount fairly |
The variance of the data is this chapter's spread. The variance of a model is how much its predictions swing when the training data changes, the thing you fight with regularization and more data. They are different quantities that share a name because both measure how much something varies around its center.
Real-World Example: Two Carriers, Same Average
The whole point of dispersion is that the average can hide everything that matters. These 300 deliveries split between two carriers make it vivid: both average about 5 days, but one is dependable and the other is a gamble, a difference the mean alone completely misses and the spread makes obvious. The companion notebook compares their SD, IQR, range, and coefficient of variation.
One row per order: order_id, carrier (SwiftShip or
ValuePost), and delivery_days, the days from dispatch to delivery. The two carriers are built to
share a mean but differ sharply in consistency, so the story lives entirely in the spread.
Bring it to life in Python
The companion notebook contrasts two same-mean datasets, builds variance and SD from scratch (n vs n−1), checks the 68-95-99.7 rule, runs the box-plot outlier test, and compares scales with the coefficient of variation.
View opens the rendered notebook instantly (no setup). Open in Colab runs &
edits it live in your browser. To run locally, install numpy, pandas, matplotlib
and launch jupyter notebook.
🎓 Key Takeaways
- ✓Dispersion measures what the mean hides: how spread out the data is.
- ✓Range is quick but fragile; the IQR (middle 50%) resists outliers and drives the box plot and the 1.5×IQR rule.
- ✓Variance squares deviations; standard deviation roots it back into real units. Use n − 1 for a sample.
- ✓68-95-99.7 makes the SD a ruler for bell-shaped data; Chebyshev gives looser limits for any shape; the z-score counts SDs from the mean.
- ✓CV compares relative spread across scales; report mean ± SD for symmetric data, median + IQR when skewed.
Practice Challenges
Five short challenges covering the chapter. Beginner to intermediate; try them on paper or in Python.
Range & IQR
For [12, 15, 14, 10, 18, 20, 13, 16], find the range, Q1,
Q3, and the IQR.
Variance & SD by hand
For [5, 7, 3, 9, 6], compute the deviations from the mean, then the sample
variance (÷ n−1) and sample standard deviation.
Use the 68-95-99.7 rule
Scores are bell-shaped with mean 500, SD 100. (a) What range holds the middle ~95%? (b) Roughly what % score above 700?
Coefficient of variation
Stock A: mean 8%, SD 4%. Stock B: mean 2%, SD 1.5%. Which is relatively more variable?
Robustness check
Start with [20, 22, 21, 23, 19], then add 200. Recompute the SD
and the IQR. Which measure is the robust one, and why?
A fully-worked solutions notebook walks through all five challenges in the same visual style. Try them yourself first, then compare.
Quiz: Test Yourself
Eight quick questions on dispersion. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.