Quality control is inference under pressure: a wrong call ships defects or halts a healthy line. This case study shows one of the most important lessons of the Part, that a single summary number can hide real problems, and how the right tests pull them into the light.
(1) Is the overall mean fill weight on the 250 g target? (2) Do the three lines differ in average weight? (3) Is the defect rate associated with shift?
The Data & The Questions
One row per sampled unit: its line, shift, measured fill_weight_g, and
whether a defect was found. The overall average weight is about 249.85 g, reassuringly
close to 250.
One row per unit with line (three lines), shift (day/night),
fill_weight_g, defect (0/1), and defect_type.
If you stopped at the overall average, you would sign off the process as healthy. That is the trap this chapter is built to expose: an aggregate on target can hide one line running light, another running heavy, and defects clustering in one shift. We test all three questions before concluding anything.
Three Questions, Three Tests
Each question's structure picks its tool: one mean vs a target, three groups compared, and a rate by category.
| Question | Structure | Test | H₀ / H₁ |
|---|---|---|---|
| On the 250 g target? | one mean vs a value | one-sample t-test | μ = 250 / μ ≠ 250 |
| Do the lines differ? | 3 independent groups, numeric | one-way ANOVA + Tukey | lines equal / a line differs |
| Defects tied to shift? | 2 categorical variables | chi-square independence | independent / associated |
The subtle point is that Q1 and Q2 are different questions. The process can be on target on average while individual lines run off target in opposite directions and cancel out. A one-sample test against the target will not catch that, only comparing the lines will. This is precisely the trap to check for.
The Analysis & Results
The one-sample test checks the aggregate; ANOVA-plus-Tukey finds the line running light; chi-square ties defects to shift.
| Question | Result | Reading |
|---|---|---|
| Q1 mean vs 250 g | 249.85 g, t = −0.98, p ≈ 0.33 | consistent with 250 g (in aggregate) |
| Q2 ANOVA across lines | F = 15.7, p ≈ 10⁻⁷ | lines differ |
| Q2 Tukey | Line-2 vs Line-1 (−1.8 g) and vs Line-3 (−1.7 g) differ; 1 ≈ 3 | Line-2 is the culprit |
| Q3 defect × shift | day 6.0% vs night 19.6%, χ² = 12.8, p ≈ 0.0003 | night is worse |
The whole lesson in one dataset. The overall mean is consistent with the target (p ≈ 0.33, 95% CI 249.54 to 250.15 g), yet the ANOVA says the lines differ (p ≈ 10⁻⁷, η² ≈ 0.09, a moderate effect) and Tukey pins it on Line-2, about 1.3 g under the 250 g target and 1.7 to 1.8 g below the other two lines. Lines 1 and 3 sit only slightly heavy (+0.50 and +0.38 g, per-line p ≈ 0.045 and 0.16), and the errors cancel in the average. Separately, defects are three times more common at night (19.6% vs 6.0%, p ≈ 0.0003). Two real, actionable problems hidden behind a healthy-looking headline.
Assumptions, checked. Before trusting the line ANOVA we check its conditions. Levene's test (p ≈ 0.79) finds no evidence of unequal variances, a residual QQ plot with Shapiro (p ≈ 0.58) finds none of non-normal residuals, and a distribution-free Kruskal-Wallis cross-check agrees. Had the variances differed, Welch's ANOVA would handle it, and with 110 units per line the F-test tolerates moderate non-normality. Kruskal-Wallis compares the lines' whole distributions, so it backs up a difference in means only when their shapes are similar.
The Statistician's Report
How to brief the plant manager, with the punchline first.
On target on average, with two problems underneath
What we found. Overall the plant average is consistent with its 250 g target, so a single headline number would say "all good." It is not. Line-2 is under-filling by about 1.3 grams against the target while Lines 1 and 3 run slightly heavy, and the over- and under-fills cancel out in the plant average. And defects are about three times more common on the night shift (about 20% vs 6%).
How confident are we? If the lines truly filled alike, a gap this large would turn up less than once in a million samples, and the careful pairwise comparison isolates Line-2. The night-shift defect difference is also clear (a gap this large would turn up about once in 3,000 samples if shift made no difference).
What to do. (1) Recalibrate Line-2's filler, which risks under-weight complaints. Lines 1 and 3 run only slightly heavy, by amounts that per-line tests can't clearly separate from 250 g once three lines are tested. (2) Investigate the night shift, staffing, fatigue, lighting, or maintenance timing, because that defect rate is the bigger cost.
Caveats. The on-target average is a warning to monitor each line separately, not just the plant total, aggregates can mask offsetting errors. This is a one-time snapshot; set up ongoing control charts to catch drift as it happens.
This is statistical process control, the ancestor of modern anomaly detection and monitoring. The "aggregate hides the problem" trap is the same one that bites dashboards and ML metrics: a flat overall accuracy can hide a subgroup the model fails on. Segment, then test.
Run the quality analysis in Python
The companion notebook explores by line and by shift first (so the hidden problems surface), then uses statsmodels (DescrStatsW, anova_lm, pairwise_tukeyhsd) for the spec test, the ANOVA, and the post-hoc, and scipy for the defect-by-shift chi-square.
View opens the rendered notebook instantly (no setup). Open in Colab runs &
edits it live in your browser. To run locally, install numpy, pandas, scipy,
matplotlib, statsmodels, and openpyxl and launch jupyter notebook.
🎓 Key Takeaways
- ✓Aggregates can lie: the plant mean is consistent with the target (p ≈ 0.33), yet the lines genuinely differ.
- ✓One mean vs a target → one-sample t; 3 lines → ANOVA + Tukey to find the culprit (Line-2).
- ✓Defects by shift → chi-square: night 19.6% vs day 6.0% (p ≈ 0.0003), a real, actionable gap.
- ✓Always segment before declaring a process healthy; monitor each line, not just the total.
- ✓Up next: the Part's last case study, a paired pre-post study, where each user serves as their own control.
Quiz: Test Yourself
Eight quick questions on this case study. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.
Across five case studies you turned raw business questions, A/B tests, group comparisons, a clinical trial, a survey, and a factory line, into the right test, run correctly and reported honestly. The Part's last case study, Case Study: A Paired Pre-Post Study, measures the same users before and after a redesign and shows what pairing adds.