The last chapter counted things, whole numbers of successes or events. But height, time, and temperature are not counts; they vary smoothly and can land on any value in a range. For these continuous random variables, the whole machinery shifts from mass to density.
This is the one idea that trips everyone up. For a continuous variable, the PDF's height is not a probability, and the probability of any single exact value is zero (an interval of zero width has zero area). Only ranges carry probability, and you get it by measuring area, which is why the CDF in section 4 does so much work.
From Mass to Density
A discrete distribution stacks probability into bars (the PMF); a continuous one smears it into a smooth curve (the PDF). The probability that X falls between a and b is the shaded area beneath the curve from a to b.
A histogram of a large sample traces out the PDF, and the fraction of data in any window matches the area there. In the notebook, P(0.5 < X < 1.5) for a standard normal comes out to 0.2417, both from the area and from counting samples. The height alone tells you nothing; you must integrate, that is, measure area.
The Uniform and Exponential
Two simple continuous distributions cover a lot of ground. The uniform spreads probability evenly across an interval, every value equally likely. The exponential models the waiting time until a random event, and it has a famous quirk.
The exponential is the continuous cousin of the geometric: both answer "how long until the first event", and both are memoryless. Having already waited tells you nothing about the wait remaining, P(X > 3 | X > 2) = P(X > 1). Its mean is 1/λ, and like the geometric it has a long right tail, so unusually long waits are uncommon but never impossible.
The Normal (Gaussian) Distribution
The normal distribution, the bell curve, is the most important continuous distribution in all of statistics. It has two parameters: the mean μ sets where the bell sits, and the standard deviation σ sets how wide it spreads.
Every normal curve is the same symmetric bell, relocated by μ and stretched by σ. A larger σ spreads the same total area of 1 over a wider range, lowering the peak. Its sweeping importance, and why it appears whenever many small effects add up, comes from the Central Limit Theorem in the Sampling Distributions & the Central Limit Theorem chapter. The Normal Distribution in Depth is devoted entirely to it.
The CDF, Quantiles, and the Gamma Family
Because probabilities are areas, we need a fast way to read them. The cumulative distribution function (CDF) gives P(X ≤ x), the area to the left of x. It climbs from 0 to 1, and running it backward turns a probability into a value, a quantile or percentile.
For the standard normal, P(X ≤ 1.5) = 0.9332, and reading across at 0.95 gives the 95th percentile, 1.645. One more family rounds out the toolkit: the gamma distribution (a sum of exponentials) flexes from sharply right-skewed to nearly bell-shaped as its shape parameter grows, which is why it models rainfall totals, insurance claims, and other positive, skewed quantities.
With four continuous shapes in hand, the practical skill is picking the right one. This table is the quick map:
| Distribution | Reach for it when | Tell-tale sign in the data |
|---|---|---|
| Uniform | every value in a range is equally likely | flat histogram; a known hard min and max |
| Exponential | time between random events, or memoryless lifetimes | right-skewed, decays from zero; mean ≈ SD |
| Normal (Gaussian) | sums or averages of many small effects; measurement error | symmetric bell; the 68-95-99.7 rule holds |
| Gamma | positive, right-skewed totals: rainfall, claims, waiting times | skewed with a hump; the shape sets how skewed |
| Log-normal (a common cousin) | multiplicative growth: incomes, city sizes, stock returns | right-skewed, but the log of it looks normal |
Continuous Distributions in Machine Learning & AI
Continuous distributions, and the Gaussian above all, are woven through machine learning. They model the data, the noise, the weights, and the uncertainty, and the one you assume quietly decides your loss function.
| Distribution | In ML / AI it appears as | Concrete example |
|---|---|---|
| Normal (Gaussian) | The default for errors, noise, and weights | Gaussian noise in diffusion models and VAEs; weight initialization |
| Normal likelihood | Squared-error (MSE) loss | least-squares regression is Gaussian maximum likelihood |
| Uniform | Random initialization and sampling | uniform weight init; sampling actions or seeds |
| Exponential / Gamma | Time-to-event and positive quantities | survival models; waiting times between events |
| Mixtures & densities | Density estimation | Gaussian mixture models; kernel density estimation |
The Gaussian is the unofficial mascot of machine learning. Assume your model's errors are normal and minimizing squared error becomes maximum likelihood, which is why MSE is the default regression loss. Diffusion models generate images by gradually removing Gaussian noise; variational autoencoders place a Gaussian over a latent space; Gaussian processes put a distribution over entire functions; and sensible weight initialization (He, Xavier) draws from carefully scaled normals to keep signals stable through deep networks. Meanwhile the exponential and gamma drive survival analysis and event modeling. Knowing a distribution's shape, mean, and tails is what lets a researcher choose the right model, and the right loss, for the data at hand.
Explore densities in Python
The companion notebook overlays a PDF on a sample histogram and shades a probability as area, plots the uniform and exponential (and checks the memoryless property), shows how μ and σ reshape the normal, reads probabilities and the 95th percentile off the CDF, surveys the gamma family, and fits a Gaussian by maximum likelihood, the calculation behind least squares.
View opens the rendered notebook instantly (no setup). Open in Colab runs &
edits it live in your browser. To run locally, install numpy, scipy, and
matplotlib and launch jupyter notebook.
🎓 Key Takeaways
- ✓Continuous variables use a PDF; probability is the area under it over an interval, and the total area is 1.
- ✓Density is not probability: the height is not a probability, and P(X = exact value) = 0. Only ranges carry probability.
- ✓Uniform is flat; exponential models memoryless waiting times with mean 1/λ.
- ✓The normal has parameters μ (center) and σ (spread); the CDF gives P(X ≤ x) and its inverse gives quantiles.
- ✓In ML/AI, Gaussian noise, MSE = Gaussian likelihood, weight init, diffusion, VAEs, and Gaussian processes all rest on these curves.
Practice Challenges
Five short challenges, beginner to intermediate. Try them on paper or with SciPy before checking the solutions.
Uniform
A bus is equally likely to arrive at any minute in a 10-minute window, X ~ Uniform(0, 10). Find P(X < 3) and the mean wait.
Exponential
Buses arrive on average every 10 minutes (exponential). Find P(wait > 15) and P(wait < 5).
Normal
IQ scores follow Normal(100, 15). Find P(X < 115) and P(85 < X < 115).
Quantile
For the same N(100, 15), find the 90th percentile, the score 90% of people fall below.
Fit a Gaussian
Given a sample of measurements, estimate the normal's parameters by maximum likelihood.
A fully-worked solutions notebook walks through all five challenges, each verified in code. Try them yourself first, then compare.
Quiz: Test Yourself
Eight quick questions on continuous distributions. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.