The Point vs. Interval Estimation chapter promised an interval; here we build it for the most common target, a population mean. The recipe is always estimate ± (critical value)·(standard error), and the only real question is which critical value to use.
We almost never know the population σ, so the t-interval is the default. The z-interval is the clean textbook starting point that shows where the t correction comes from.
The z-Interval (σ known)
If the population standard deviation were known, the 95% interval is simply x̄ ± 1.96·(σ/√n). It is clean and exact, but it leans on a quantity we rarely have.
Plugging the sample standard deviation s into this z-formula understates the uncertainty for small samples, because s is itself a noisy estimate of σ. The proper fix is to widen the interval using the t-distribution.
The t-Interval (σ unknown)
Replace σ with the sample sd s, and replace z with t*, the critical value of Student's t with n−1 degrees of freedom. The t-distribution has heavier tails, so t* > z, and the interval is a little wider, the honest price of estimating σ.
| Degrees of freedom (n−1) | t* (95%) | z* (95%) |
|---|---|---|
| 4 | 2.776 | 1.960 |
| 9 | 2.262 | 1.960 |
| 29 | 2.045 | 1.960 |
| 99 | 1.984 | 1.960 |
| 1000 | 1.962 | 1.960 |
At df = 4 the critical value is 2.78, far above 1.96; by n ≈ 30–60 the gap is
negligible, and for large samples t* and z* are interchangeable. In Python it is one line:
scipy.stats.t.interval(0.95, df=n-1, loc=xbar, scale=s/√n).
Conditions & Coverage
A confidence interval is only as good as its assumptions. The t-interval needs the data to be roughly normal, or the sample to be large enough that the Central Limit Theorem makes x̄ approximately normal.
| Sample size n | Simulated coverage (skewed data) | Verdict |
|---|---|---|
| 10 | 93.5% | under-covers; CLT has not kicked in |
| 30 | 94.0% | close; the usual rule-of-thumb threshold |
| 100 | 94.7% | on target |
The notebook draws thousands of intervals from a right-skewed population. With n = 10 the interval under-covers, but by n = 30 coverage is near the promised 95% and improves further as n grows. The practical rule: trust the t-interval when the data is roughly symmetric, or when n ≥ ~30. For heavily skewed data and small samples, prefer the bootstrap of Resampling & Simulation.
Real-World Example: Mean Delivery Time
A logistics team exports 180 completed shipments and wants to report the average door-to-door delivery time with quantified confidence. σ is unknown, so this is a textbook t-interval, after a quick shape check.
Each row is a shipment with delivery_hours, distance_mi,
warehouse, and carrier.
| Quantity | Value |
|---|---|
| Sample size n | 180 shipments |
| Mean delivery time (x̄) | 35.85 hours |
| Sample sd (s) | 10.11 hours |
| Standard error (s/√n) | 0.753 hours |
| t* (df = 179) | 1.973 |
| 95% confidence interval | 34.37 to 37.34 hours |
The histogram is reasonably symmetric and n = 180 is large, so the conditions hold. The team can now state, with 95% confidence, that the true average delivery time is between about 34.4 and 37.3 hours, "roughly a day and a half, give or take an hour", instead of a single unqualified number. That interval is the difference between a defensible service-level claim and a guess.
Confidence Intervals in Machine Learning & AI
The t-interval is the everyday tool for putting error bars on any average a model or experiment produces, from mean latency to mean reward.
| Idea (this chapter) | In ML / AI it becomes | Example |
|---|---|---|
| t-interval for a mean | Error bars on a mean metric | mean inference latency ± CI |
| Degrees of freedom | How many independent runs you have | 5 cross-validation folds → df = 4 |
| Conditions / CLT | When a normal approximation is safe | average over enough batches/episodes |
| t vs z | Small-sample honesty | few seeds → wider intervals (don't overclaim) |
Reporting "mean reward over 5 seeds" without an interval is one of the most common reproducibility failures in ML. With only 5 runs, df = 4 and t* = 2.78, so the interval is much wider than a normal approximation would suggest, and two methods whose intervals overlap are not distinguishable. The t-interval is exactly the right tool: it widens automatically when you have few runs, protecting you from declaring a winner on noise.
Build z- and t-intervals in Python
The companion notebook builds the z-interval, derives the t-interval and tabulates how t* shrinks toward z*,
checks coverage on skewed data, and loads confidence-intervals-for-a-mean--delivery_times.xlsx to put a 95% interval on the
mean delivery time with scipy.stats.t.interval.
View opens the rendered notebook instantly (no setup). Open in Colab runs &
edits it live in your browser. To run locally, install numpy, pandas, scipy,
matplotlib, and openpyxl and launch jupyter notebook.
🎓 Key Takeaways
- ✓CI for a mean = x̄ ± (critical value)·SE, with SE = σ/√n (or s/√n).
- ✓σ known → z-interval (z = 1.96 for 95%); σ unknown → t-interval with df = n−1.
- ✓t* > z* (heavier tails), converging to z as n grows; 2.78 at df=4, ~1.96 by df=1000.
- ✓Conditions: roughly normal data, or n ≥ ~30 for the CLT; skewed + small n under-covers.
- ✓Real data: mean delivery time 35.9 h with a 95% interval of 34.4–37.3 h; in ML, report metric intervals (few seeds → wider).
Practice Challenges
Five short challenges, beginner to intermediate. Try them with NumPy and SciPy before checking the solutions.
z-interval (σ known)
With σ = 8 known, build a 95% z-interval for the mean from a sample of 40 drawn from Normal(50, 8).
t-interval (σ unknown)
Now treat σ as unknown: use the sample sd and stats.t.interval for the same sample.
t* shrinks toward z*
Tabulate the 95% t critical value for df = 4, 9, 29, 99 and compare to z* = 1.96.
stats.t.ppf(0.975, df).Coverage check
From a skewed Gamma population, confirm the t-interval coverage is near 95% at n = 40.
Real data: mean delivery time
Load confidence-intervals-for-a-mean--delivery_times.xlsx and report the 95% t-interval for the mean delivery time.
pd.read_excel(..., sheet_name="Shipments").A fully-worked solutions notebook walks through all five challenges, each verified in code. Try them yourself first, then compare.
Quiz: Test Yourself
Eight quick questions on confidence intervals for a mean. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.
Means are not the only target. The Confidence Intervals for Proportions & Differences chapter builds intervals for proportions and differences, the foundation of polling and A/B testing, where the interval for a difference tells you whether two groups truly differ.