Heights, measurement errors, test scores, the average of almost anything, all tend to pile up into the same symmetric mound. The normal distribution is so central to statistics that it earns a chapter of its own, and a fluency that the rest of the book assumes.
The normal's dominance is not a coincidence. The Central Limit Theorem (see Sampling Distributions & the CLT) shows that sums and averages of many independent influences become normal, whatever their original distribution. Since so much of the world is the accumulation of small random effects, the bell curve turns up again and again, and the tools in this chapter apply to all of it.
The Bell Curve and Its Two Parameters
A normal distribution is fixed by just two numbers. The mean μ locates the peak; the standard deviation σ sets the width. Because the curve is perfectly symmetric, the mean, median, and mode all coincide at the center.
Every normal is this same bell, relocated by μ and stretched by σ. A smaller σ concentrates the area into a tall, narrow spike; a larger σ flattens and widens it, since the total area must always equal 1. Knowing μ and σ tells you everything about the distribution.
The Empirical Rule: 68 - 95 - 99.7
The most useful fact about the normal is how its area is distributed around the mean. For any normal, the same fixed fractions fall within 1, 2, and 3 standard deviations, the empirical rule.
The exact figures are 68.27%, 95.45%, and 99.73%, and they hold for every normal regardless of μ and σ. This is why "within two standard deviations" is shorthand for "almost everyone", and why a value beyond 3σ is genuinely rare, under 0.3% of cases. A simulation of a million draws reproduces these percentages to two decimals.
The Standard Normal and z-Scores
To compare values measured on different scales, convert each to a z-score: how many standard deviations it sits from its mean, z = (x − μ) / σ. This transformation turns any normal into the standard normal N(0, 1).
A z-score of +2 means "two standard deviations above average", whether the raw value was a test score, a height, or a temperature. Standardizing a dataset gives it mean 0 and standard deviation 1 by construction, so two values from totally different scales become directly comparable, the heart of fair comparison in statistics.
Probabilities and Percentiles with z
Once a value is a z-score, its probability comes straight from the standard normal CDF, historically a printed z-table, now a one-line function. Convert to z, look up the area, and you have the probability or percentile.
| Question | In z | Answer |
|---|---|---|
| Fraction below 1σ above the mean | P(Z < 1) | 0.8413 |
| Fraction beyond 2σ above the mean | P(Z > 2) | 0.0228 |
| z-range holding the central 95% | ±z for 0.975 | ±1.96 |
| 99th percentile of IQ, N(100, 15) | μ + 2.326σ | 134.9 |
The single most-used number in applied statistics is z = 1.96: the central 95% of any normal lies within ±1.96 standard deviations of the mean. It is the engine behind 95% confidence intervals and the 0.05 significance level you will meet throughout the inference chapters. Many people round it to 2, which is the empirical rule's 95%.
The Normal Distribution in Machine Learning & AI
The z-score is not just for textbooks; standardizing features is the single most common preprocessing step in machine learning, and the normal distribution underpins how models are initialized, regularized, and generate data.
| Normal idea | In ML / AI it becomes | Concrete example |
|---|---|---|
| z-score standardization | Feature scaling (StandardScaler) | rescale every feature to mean 0, std 1 before training |
| Empirical rule / |z| > 3 | Anomaly and outlier detection | flag points more than 3σ from the mean |
| Gaussian weights | Network initialization | He / Xavier init draw from scaled normals |
| Normalizing activations | Batch / layer normalization | standardize activations to stabilize training |
| Gaussian noise | Generative models | diffusion and VAEs sample from N(0, 1) |
Before a model sees your data, the data is almost always standardized: each feature converted to a z-score so that income (in tens of thousands) and age (in tens) contribute on equal footing. Without it, distance-based and gradient-based methods are dominated by whichever feature has the biggest raw numbers. The same z-score is a ready-made anomaly detector (|z| > 3 is a rare outlier by the empirical rule). Deeper in the stack, careful Gaussian weight initialization (He, Xavier) keeps signals from exploding or vanishing through dozens of layers, batch normalization standardizes activations to speed training, and diffusion models and VAEs are built directly on sampling from N(0, 1). The humble bell curve is quietly load-bearing across modern AI.
Work the bell curve in Python
The companion notebook draws the bell for several μ and σ, verifies the 68-95-99.7 rule by
simulating a million draws, standardizes scores into z-scores, reads probabilities and the 1.96 cutoff off
the standard normal, and standardizes two features on wildly different scales, the exact preprocessing step
behind StandardScaler, while flagging 3σ anomalies.
View opens the rendered notebook instantly (no setup). Open in Colab runs &
edits it live in your browser. To run locally, install numpy, scipy, and
matplotlib and launch jupyter notebook.
🎓 Key Takeaways
- ✓The normal is the symmetric bell set by μ (center) and σ (spread); mean, median, and mode coincide.
- ✓Empirical rule: 68% within 1σ, 95% within 2σ, 99.7% within 3σ, for every normal.
- ✓z-scores standardize any value to N(0, 1): z = (x − μ)/σ, making different scales comparable.
- ✓Probabilities and percentiles come from the standard normal CDF; the central 95% lies within ±1.96σ.
- ✓In ML/AI, z-score standardization, anomaly detection, Gaussian init, batch norm, and diffusion all build on the normal.
Practice Challenges
Five short challenges, beginner to intermediate. Try them on paper or with SciPy before checking the solutions.
Empirical rule
Adult heights are Normal(170 cm, 8 cm). What fraction of people are between 162 and 178 cm? Between 154 and 186 cm?
z-score
On a test with mean 75 and standard deviation 5, a student scores 85. Find the z-score and the percentile.
Compare via z
Ana scored 88 on Exam A (mean 80, sd 4); Ben scored 90 on Exam B (mean 85, sd 10). Whose result is more impressive relative to peers?
Tail probability
IQ scores follow Normal(100, 15). What fraction of people score above 130?
Standardize a dataset
Standardize [10, 12, 14, 16, 18] to z-scores, the way a model would before training.
A fully-worked solutions notebook walks through all five challenges, each verified in code. Try them yourself first, then compare.
Quiz: Test Yourself
Eight quick questions on the normal distribution. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.