The CLT made sample means normal, which lets us test them. But there is a catch: we almost never know the population standard deviation σ, so we estimate it with the sample s. That extra uncertainty is exactly what the t-distribution accounts for.
biomedical_clinical_trials.csv records bp_change_mmHg for 1,000 patients split into
an Active_Compound_X arm and a Placebo arm. We test whether blood pressure changed,
and whether the drug actually beat placebo.
The One-Sample t-Test
The obvious first question: did blood pressure drop in the treatment arm? We test H0: mean bp change = 0. In the Active arm, BP fell by about 2.6 mmHg, and the t-statistic is a whopping −14.2 (df = 484), with a p-value near 10−38.
The t-statistic (x̄ − 0)/(s/√n) uses the sample standard deviation s, which is precisely why the t-distribution, not the normal, is the honest reference. The result is unambiguous: blood pressure fell in the treatment arm. Case closed, the drug works? Not so fast.
The Placebo Trap
Efficacy means beating placebo, not just changing from baseline. Blood pressure drifts on its own, regression to the mean, the placebo effect, seasonal change. The two-sample t-test compares the arms directly, and the verdict is sobering.
The placebo arm dropped just as much, −2.74 vs −2.58 mmHg, and the two-sample t-test returns p ≈ 0.52 with an effect size of Cohen's d ≈ 0.04 (essentially zero): no significant difference, and no meaningful one either. The improvement in section 1 was real but not caused by the drug; BP fell in both groups. This is the whole reason trials are placebo-controlled. Testing against zero confuses a real-but-spurious drop with genuine efficacy; only the comparison reveals the truth.
Why t, Not z?
The t looks like the normal but with heavier tails that fatten as the sample shrinks. Its critical value exceeds the normal's 1.96, demanding stronger evidence when σ is estimated from few data points.
| sample size n | df = n − 1 | 95% t critical | normal z |
|---|---|---|---|
| 5 | 4 | 2.78 | 1.96 |
| 10 | 9 | 2.26 | 1.96 |
| 15 | 14 | 2.14 | 1.96 |
| 30 | 29 | 2.04 | 1.96 |
| 100 | 99 | 1.98 | 1.96 |
At n = 5 the 95% critical value is 2.78, far above 1.96, the t's way of demanding more evidence when σ rests on few points. As n grows the gap closes, and by n = 100 the t is essentially the normal. Using z on a tiny trial would overstate significance; the t keeps small-sample conclusions honest.
Run the trial analysis
The companion notebook runs the one-sample t-test on the treatment arm (a huge, significant drop), then the two-sample t-test against placebo (no significant difference), with side-by-side boxplots, and tabulates how the t critical value shrinks toward z as the sample grows.
View opens the rendered notebook instantly (no setup). Open in Colab runs &
edits it live in your browser. To run locally, install numpy, pandas, scipy,
and matplotlib and launch jupyter notebook.
🎓 Key Takeaways
- ✓The t-distribution tests a mean when σ is unknown; the statistic is (x̄ − μ0)/(s/√n), with df = n − 1.
- ✓One-sample t-test: BP dropped 2.6 mmHg in the treatment arm, t = −14.2, p ≈ 10−38, clearly significant.
- ✓The placebo trap: placebo dropped just as much; the two-sample test gave p ≈ 0.52, no real drug effect.
- ✓Compare to a control, not to zero, or you mistake a spurious drift for efficacy.
- ✓t has heavier tails than z (critical 2.78 at n = 5), converging to the normal as n grows.
Quiz: Test Yourself
Eight quick questions on the t-test case study. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.