A die lands on 6 a few too many times. Email is the top support channel, but only in some regions. These are questions about counts, not means, and the chi-square test answers them by comparing observed counts to the counts expected under a null hypothesis.
Compare observed counts to expected counts. Big gaps → big χ² → small p. Report Cramer's V so a tiny p from a huge sample is not mistaken for a strong relationship.
The Chi-Square Idea: Observed vs Expected
Every chi-square test rests on one comparison. For each category, take the observed count O and the count E we would expect if H₀ were true, and measure the gap with (O − E)² / E. Summing across categories gives a single number that is small when the data behave and large when they surprise.
The contribution (O−E)²/E is tiny where the data match expectation and grows where they diverge. Under H₀ this sum follows a chi-square distribution, whose degrees of freedom depend on the design, which is what turns the raw discrepancy into a p-value.
Goodness-of-Fit
The goodness-of-fit test asks whether one categorical variable follows a hypothesized distribution, a fair die (equal proportions), a claimed market split, last year's mix. Degrees of freedom = (number of categories − 1).
In the notebook, observed tier counts of 330/290/180/100 against a claimed 40/30/20/10 split give χ² ≈ 5.1 (p ≈ 0.17), consistent with the claim. The test is the categorical analog of a one-sample test: it compares a whole distribution of counts to a benchmark. Always check that every expected count is at least about 5, or the approximation wobbles.
Test of Independence & Cramer's V
The test of independence works on a contingency table, rows for one categorical variable, columns for another, and asks whether they are related. The expected count for a cell, assuming independence, is (row total × column total) / grand total. Degrees of freedom = (rows − 1)(columns − 1).
A significant chi-square says the two variables are associated, the categorical cousin of correlation. But with enough data even a negligible association turns significant, so report Cramer's V, a rescaling of χ² to a 0-to-1 effect size. V near 0 means barely related; V near 1 means tightly linked. In the notebook a 2×3 table gives χ² ≈ 25 (p ≈ 0.000) with V ≈ 0.25, a moderate link.
Real-World Example: Support Channel by Region
A company logs each customer's region and preferred support channel (Email, Phone, Chat, App). Two questions, two chi-square tests: are the four channels used equally often (goodness-of-fit), and does channel preference depend on region (independence)?
One row per customer with region and the
preferred_channel (four categories).
| Question | Test | Result | Verdict |
|---|---|---|---|
| Channels used equally? | goodness-of-fit (df = 3) | χ² ≈ 13.7, p ≈ 0.003 | no, usage is uneven |
| Preference depends on region? | independence (df = 9) | χ² ≈ 81, p ≈ 10⁻¹³ | yes, associated |
| How strong is the link? | Cramer's V | ≈ 0.17 | modest, not dominant |
Both tests reject their nulls. The four channels are not used equally (goodness-of-fit p ≈ 0.003), and channel preference clearly depends on region (independence p ≈ 10⁻¹³). Yet the effect size tells the fuller story: Cramer's V ≈ 0.17 means the association, though unmistakably real given 900 customers, is modest, region shifts the channel mix rather than determining it. This is the discipline the chapter keeps returning to: a microscopic p-value confirms an effect exists, but only the effect size says whether it matters.
Chi-Square in Machine Learning & AI
The chi-square statistic shows up wherever categorical features and class labels meet.
| Idea (this chapter) | In ML / AI it becomes | Example |
|---|---|---|
| Test of independence | Categorical feature selection | chi2 in scikit-learn's SelectKBest |
| Observed vs expected | Detecting data / label drift | this month's category mix vs baseline |
| Goodness-of-fit | Checking class balance | do label counts match the design? |
| Cramer's V | Strength of categorical association | which features actually relate to the target |
Chi-square is a standard feature-selection tool for categorical predictors: scikit-learn's
chi2 score ranks features by how strongly they associate with the target, exactly a test of
independence per feature. It is also a workhorse for drift detection, comparing the
category distribution of incoming data against a training-time baseline flags when the world has shifted under
a deployed model. As always, pair the test with an effect size (Cramer's V): at web scale, everything is
"significant", so the size of the association is what guides which features and which drifts deserve action.
Run both chi-square tests in Python
The companion notebook computes the statistic by hand, runs goodness-of-fit with chisquare and
independence with chi2_contingency, derives Cramer's V, and loads
chi-square-tests--customer_channel.xlsx to test channel usage and its dependence on region.
View opens the rendered notebook instantly (no setup). Open in Colab runs &
edits it live in your browser. To run locally, install numpy, pandas, scipy,
matplotlib, statsmodels, and openpyxl and launch jupyter notebook.
🎓 Key Takeaways
- ✓Chi-square compares observed to expected counts: χ² = Σ(O − E)²/E, large when they diverge.
- ✓Goodness-of-fit tests one variable vs a claimed split (df = categories − 1).
- ✓Independence tests whether two categorical variables are related (df = (r−1)(c−1)); keep expected counts ≥ ~5.
- ✓Real data: channels used unequally (p ≈ 0.003) and depend on region (p ≈ 10⁻¹³), but Cramer's V ≈ 0.17 says modestly.
- ✓In ML/AI: chi-square powers categorical feature selection and drift detection, paired with Cramer's V.
Practice Challenges
Five short challenges, beginner to intermediate. Try them before checking the solutions.
Goodness-of-fit for a die
Test whether [18, 22, 16, 20, 14, 10] (100 rolls) is consistent with a fair die.
Goodness-of-fit vs a claim
Observed [330, 290, 180, 100]; claimed split 40/30/20/10. Test it.
Test of independence
For the table [[90, 60, 50], [40, 80, 70]], test independence and report the degrees of freedom.
stats.chi2_contingency.Cramer's V
For the table above, compute Cramer's V as an effect size.
Real data: channel by region
Load chi-square-tests--customer_channel.xlsx and test independence of region and preferred_channel.
pd.crosstab first.A fully-worked solutions notebook walks through all five challenges, each verified in code. Try them yourself first, then compare.
Quiz: Test Yourself
Eight quick questions on chi-square tests. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.
z, t, F, and chi-square all lean on distributional assumptions. Nonparametric Tests handles skewed, ordinal, or outlier-ridden data by testing ranks instead of raw values.