The t-test compared means. But much of the world is categorical: tier, region, satisfaction, yes/no. To ask whether two such variables are related, we count, and the chi-square test of independence turns those counts into a verdict.
ab_survey_demographics.csv holds 1,000 survey responses with subscription_tier
(Free / Standard / Premium), demographic_region, and satisfaction_level
(Low / Medium / High). We test which of these is genuinely linked to satisfaction.
The Contingency Table
Chi-square works on counts, not averages. We cross-tabulate every respondent's tier against their satisfaction level. The result is a contingency table, the raw material of the test.
| tier ↓ / satisfaction → | Low | Medium | High |
|---|---|---|---|
| Free | 202 | 198 | 101 |
| Standard | 60 | 157 | 82 |
| Premium | 17 | 53 | 130 |
Premium users pile up in High satisfaction (130) while Free users pile up in Low (202). The pattern looks real, but is it more than sampling noise? That is precisely the question chi-square is built to answer.
Observed vs Expected
If tier and satisfaction were independent, each cell's count would be (row total × column total) / grand total. Chi-square measures how far reality strays from that.
The gaps are stark. Premium / Low has just 17 respondents where independence predicts 56; Free / Low has 202 where it predicts 140. Summing (O − E)²/E across all nine cells gives
χ² = 175.4, df = (3−1)(3−1) = 4, p ≈ 10−37
The tiny p-value says the link is real. Cramer's V ≈ 0.30 says it is moderately strong, not just detectable. Always report an effect size alongside p: with 1,000 respondents even trivial associations can become significant.
The Control: Region
Significance is not automatic. A good test also confirms independence when there is nothing there. We run
the identical test on demographic_region against satisfaction.
χ² = 4.8, df = 4, p ≈ 0.31
Here the statistic is tiny and p ≈ 0.31 sits far above 0.05, so we fail to reject independence: satisfaction does not vary by region. This is the discipline of the chi-square test, it flags the genuine association (tier) and clears the spurious one (region), rather than finding a pattern in everything.
Chi-square needs expected counts of at least ~5 per cell, and "fail to reject" means no evidence of a link, not proof of independence. Absence of evidence is not evidence of absence.
Run the survey analysis
The companion notebook builds the contingency table, runs scipy.stats.chi2_contingency on tier
vs satisfaction (significant) and region vs satisfaction (not), prints the expected-count matrix, and
computes Cramer's V for effect size.
View opens the rendered notebook instantly (no setup). Open in Colab runs &
edits it live in your browser. To run locally, install numpy, pandas, scipy,
and matplotlib and launch jupyter notebook.
🎓 Key Takeaways
- ✓Chi-square tests independence of two categorical variables from a contingency table of counts.
- ✓χ² = Σ(O − E)²/E, where E = (row total × col total)/grand total; df = (r−1)(c−1).
- ✓Tier × satisfaction: χ² = 175, p ≈ 10−37, a real, moderate link (Cramer's V ≈ 0.30).
- ✓Region × satisfaction: χ² = 4.8, p ≈ 0.31, fail to reject, independent.
- ✓Report effect size, not just p; and keep expected cell counts ≥ ~5 for the test to be valid.
Quiz: Test Yourself
Eight quick questions on the chi-square case study. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.