Quality control is inference under pressure: a wrong call ships defects or halts a healthy line. This final case study shows the most important lesson of the Part, that a single summary number can hide real problems, and how the right tests pull them into the light.
(1) Is the overall mean fill weight on the 250 g target? (2) Do the three lines differ in average weight? (3) Is the defect rate associated with shift?
The Data & The Questions
One row per sampled unit: its line, shift, measured fill_weight_g, and
whether a defect was found. The overall average weight is about 249.85 g, reassuringly
close to 250.
One row per unit with line (three lines), shift (day/night),
fill_weight_g, defect (0/1), and defect_type.
If you stopped at the overall average, you would sign off the process as healthy. That is the trap this chapter is built to expose: an aggregate on target can hide one line running light, another running heavy, and defects clustering in one shift. We test all three questions before concluding anything.
Three Questions, Three Tests
Each question's structure picks its tool: one mean vs a target, three groups compared, and a rate by category.
| Question | Structure | Test | H₀ / H₁ |
|---|---|---|---|
| On the 250 g target? | one mean vs a value | one-sample t-test | μ = 250 / μ ≠ 250 |
| Do the lines differ? | 3 independent groups, numeric | one-way ANOVA + Tukey | lines equal / a line differs |
| Defects tied to shift? | 2 categorical variables | chi-square independence | independent / associated |
The subtle point is that Q1 and Q2 are different questions. The process can be on target on average while individual lines drift in opposite directions and cancel out. A one-sample test against the target will not catch that, only comparing the lines will. This is precisely the trap to check for.
The Analysis & Results
The one-sample test checks the aggregate; ANOVA-plus-Tukey finds the drifting line; chi-square ties defects to shift.
| Question | Result | Reading |
|---|---|---|
| Q1 mean vs 250 g | 249.85 g, t = −0.98, p ≈ 0.33 | on target (in aggregate) |
| Q2 ANOVA across lines | F = 15.7, p ≈ 10⁻⁷ | lines differ |
| Q2 Tukey | Line-2 vs Line-1 (−1.8 g) and vs Line-3 (−1.7 g) differ; 1 ≈ 3 | Line-2 is the culprit |
| Q3 defect × shift | day 6.0% vs night 19.6%, χ² = 12.8, p ≈ 0.0003 | night is worse |
The whole lesson in one dataset. The overall mean is statistically on target (p ≈ 0.33), yet the ANOVA says the lines differ (p ≈ 10⁻⁷, η² ≈ 0.09, a moderate effect) and Tukey pins it on Line-2, running about 1.7 g light while Lines 1 and 3 sit slightly heavy, the errors cancel in the average. Separately, defects are three times more common at night (19.6% vs 6.0%, p ≈ 0.0003). Two real, actionable problems hidden behind a healthy-looking headline.
Assumptions, checked. Before trusting the line ANOVA we confirm its conditions: Levene's test (p ≈ 0.79) for equal variance and a residual QQ plot with Shapiro (p ≈ 0.58) for normal residuals, both pass, and a distribution-free Kruskal-Wallis cross-check agrees. Had a check failed, the remedy is Welch's ANOVA for unequal variance or Kruskal-Wallis for non-normality.
The Statistician's Report
How to brief the plant manager, with the punchline first.
Findings: on-target on average, but two real problems
What we found. Overall the line hits its 250 g target on average, so a single headline number would say "all good." It is not. Line-2 is under-filling by about 1.7 grams while Lines 1 and 3 run slightly heavy, the over- and under-fills cancel out in the plant average but are real. And defects are about three times more common on the night shift (about 20% vs 6%).
How confident are we? The line difference is highly unlikely to be chance (less than 1 in a million), and the careful pairwise comparison isolates Line-2. The night-shift defect difference is also clear (about 1 in 3,000 by chance).
What to do. (1) Recalibrate Line-2's filler, it risks under-weight complaints, while the heavy lines give away product. (2) Investigate the night shift, staffing, fatigue, lighting, or maintenance timing, because that defect rate is the bigger cost.
Caveats. The on-target average is a warning to monitor each line separately, not just the plant total, aggregates can mask offsetting drift. This is a one-time snapshot; set up ongoing control charts to catch drift as it happens.
This is statistical process control, the ancestor of modern anomaly detection and monitoring. The "aggregate hides the problem" trap is the same one that bites dashboards and ML metrics: a flat overall accuracy can hide a subgroup the model fails on. Segment, then test.
Run the quality analysis in Python
The companion notebook explores by line and by shift first (so the hidden problems surface), then uses statsmodels (DescrStatsW, anova_lm, pairwise_tukeyhsd) for the spec test, the ANOVA, and the post-hoc, and scipy for the defect-by-shift chi-square.
View opens the rendered notebook instantly (no setup). Open in Colab runs &
edits it live in your browser. To run locally, install numpy, pandas, scipy,
matplotlib, statsmodels, and openpyxl and launch jupyter notebook.
🎓 Key Takeaways
- ✓Aggregates can lie: the plant mean is on target (p ≈ 0.33), yet the lines genuinely differ.
- ✓One mean vs a target → one-sample t; 3 lines → ANOVA + Tukey to find the culprit (Line-2).
- ✓Defects by shift → chi-square: night 19.6% vs day 6.0% (p ≈ 0.0003), a real, actionable gap.
- ✓Always segment before declaring a process healthy; monitor each line, not just the total.
- ✓Inference Case Studies complete: five case studies, raw question to statistician's report, the whole inference playbook in action.
Quiz: Test Yourself
Eight quick questions on this case study. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.
Across five case studies you turned raw business questions, A/B tests, group comparisons, a clinical trial, a survey, and a factory line, into the right test, run correctly and reported honestly. Next the book turns to relationships between variables: Correlation & Association, starting with covariance.