The normal distribution is the star of inference, but it assumes you know the true standard deviation σ, which you almost never do. Estimating σ from data, comparing variances, or testing categorical counts each calls for a distribution purpose-built for the job. Meet the three workhorses.
Each of these distributions is indexed by its degrees of freedom (df), loosely, the number of independent pieces of information available after estimating other quantities. Estimating the mean from a sample of n "uses up" one piece, leaving n − 1 degrees of freedom, which is why so many formulas divide by n − 1. The df controls each distribution's exact shape.
Student's t: When σ Is Unknown
When you replace the unknown population σ with the sample standard deviation s, you introduce extra uncertainty. The t distribution accounts for it: bell-shaped like the normal but with heavier tails, and it converges to the normal as the sample grows.
Heavier tails mean the t demands stronger evidence: its 97.5% critical value is 2.571 at df = 5 and 2.228 at df = 10, versus the normal's 1.96. By df = 30 the gap nearly vanishes, and by df = 100 the t is essentially the normal. This is why the t-test is the default for a mean whenever the sample is small and σ is estimated.
The Chi-Square Distribution
Square a standard normal and add several together, and you get the chi-square distribution. With k degrees of freedom it is the sum of k squared standard normals, so it is positive, right-skewed, and has mean equal to k.
Because it measures accumulated squared deviations, chi-square is the distribution for tests about variance and, most famously, for categorical data: the goodness-of-fit test (do observed counts match expected?) and the test of independence (are two categorical variables related?). In the notebook, 120 die rolls give a chi-square statistic of 6.80 (df = 5, p = 0.236), consistent with a fair die.
The F Distribution and the Family Tree
Skim-friendly: on a first read you can take the family tree below as a picture and move on to the next chapter, the algebra of how each distribution is built from normals is here for when you want the "why," but you lose nothing essential by skipping it now.
The F distribution is a ratio of two chi-squares, each divided by its degrees of freedom. It is the natural tool for comparing two variances and is the engine of ANOVA (analysis of variance, comparing several group means at once). And it completes a tidy family tree: every distribution here descends from the normal.
The relationships are exact: Z² is a chi-square with 1 df; a t is a normal divided by the square root of a scaled chi-square; and an F is the ratio of two scaled chi-squares. Understand the normal, and these three follow as consequences rather than new mysteries.
Which Distribution for Which Test?
In practice you pick the distribution by the question you are asking. This guide covers the great majority of the tests you will meet in the inference chapters ahead.
| Question | Distribution | Typical test |
|---|---|---|
| Is a mean different from a value (small sample)? | t | one-sample t-test |
| Do two groups have different means? | t | two-sample / paired t-test |
| Do observed counts match expected? | chi-square | goodness-of-fit test |
| Are two categorical variables related? | chi-square | test of independence |
| Do several groups have equal means or variances? | F | ANOVA / variance ratio |
Every one of these tests follows the same recipe: compute a statistic, locate it on the appropriate distribution (set by the degrees of freedom), and read off a p-value, the probability of a result this extreme if nothing is going on. The distributions in this chapter are simply the rulers against which test statistics are measured. The full mechanics arrive in the inference Part.
Distributions for Inference in Machine Learning & AI
These distributions are not just for classical statistics; they are how careful practitioners decide whether a model is really better, whether a feature really matters, and whether an A/B test result is real rather than noise.
| Distribution | In ML / AI it appears as | Concrete example |
|---|---|---|
| t distribution | Comparing two models | paired t-test on cross-validation fold scores |
| t distribution | A/B testing | is variant B's conversion lift significant? |
| Chi-square | Feature selection & fit | chi-square test between a categorical feature and the label |
| F distribution | Comparing many configs | ANOVA across hyperparameter groups |
| All three | Honest evaluation | turning a score gap into a p-value, not a guess |
A new model that scores 1% higher might be a genuine improvement, or it might be the luck of one train/test split. The disciplined answer is a significance test. A paired t-test across cross-validation folds asks whether one model reliably beats another (in the notebook, a 3-point gain is significant with p far below 0.05). Chi-square tests power classic feature selection (does this categorical feature carry information about the label?) and validate that generated data matches a target distribution. A/B tests behind every product decision are t-tests or proportion tests in disguise. Treating a single benchmark number as truth, with no distribution behind it, is how teams fool themselves; these three distributions are the antidote.
Run real tests in Python
The companion notebook plots the t's heavy tails and its shrinking critical values, builds the chi-square and t from squared and scaled normals to prove the family tree, runs a chi-square goodness-of-fit test on a die, and finishes with a paired t-test comparing two models across cross-validation folds, the rigorous way to claim one model beats another.
View opens the rendered notebook instantly (no setup). Open in Colab runs &
edits it live in your browser. To run locally, install numpy, scipy, and
matplotlib and launch jupyter notebook.
🎓 Key Takeaways
- ✓The t distribution handles means when σ is unknown: heavier tails than the normal, converging to it as df grows.
- ✓Chi-square is a sum of squared normals (mean = df); it tests variances and categorical counts (goodness-of-fit, independence).
- ✓The F distribution is a ratio of two chi-squares; it compares variances and powers ANOVA.
- ✓One family: Z² is chi-square(1), t = Z/√(χ²/df), F = ratio of χ²'s; all descend from the normal, indexed by degrees of freedom.
- ✓In ML/AI: paired t-tests compare models, chi-square drives feature selection, and A/B tests are t-tests in disguise.
Practice Challenges
Five short challenges, beginner to intermediate. Try them with SciPy before checking the solutions.
t critical value
For a 95% two-sided test with n = 10 (df = 9), find the critical t value and compare it to the normal's 1.96.
Chi-square mean
A chi-square is the sum of df squared standard normals. Confirm by simulation that chi-square(5) has mean 5.
t confidence interval
A sample of n = 10 has mean 50 and sample sd 8. Build a 95% confidence interval for the mean using the t.
Goodness of fit
Candy should be 25% of each of 4 colors. In 200 candies you observe [40, 55, 52, 53]. Test whether the colors are equally likely.
Compare two models
Two models are scored on 8 cross-validation folds. Use a paired t-test to decide whether model B is significantly better.
scipy.stats.ttest_rel(B, A); check p < 0.05.A fully-worked solutions notebook walks through all five challenges, each verified in code. Try them yourself first, then compare.
Quiz: Test Yourself
Eight quick questions on the distributions for inference. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.
You now have the full distribution toolkit: discrete and continuous families, the normal in depth, the sampling distribution and CLT, and the t, chi-square, and F for inference. Next, the advanced track opens with Mathematical Statistics, before the book turns to estimation and hypothesis testing.