A die lands on 6 a few too many times. Email is the top support channel, but only in some regions. These are questions about counts, not means, and the chi-square test answers them by comparing observed counts to the counts expected under a null hypothesis.
Compare observed counts to expected counts. Big gaps → big χ² → small p. Report Cramer's V so a tiny p from a huge sample is not mistaken for a strong relationship.
The Chi-Square Idea: Observed vs Expected
Every chi-square test rests on one comparison. For each category, take the observed count O and the count E we would expect if H₀ were true, and measure the gap with (O − E)² / E. Summing across categories gives a single number that is small when the data behave and large when they surprise.
The contribution (O−E)²/E is tiny where the data match expectation and grows where they diverge. Under H₀ this sum approximately follows a chi-square distribution, whose degrees of freedom depend on the design, and that is what turns the raw discrepancy into a p-value. The approximation needs independent observations, each counted as a raw count in exactly one cell, and expected counts that are not too small.
Goodness-of-Fit
The goodness-of-fit test asks whether one categorical variable follows a hypothesized distribution, a fair die (equal proportions), a claimed market split, last year's mix. Degrees of freedom = (number of categories − 1), less one more for each parameter estimated from the data (fitting a Poisson mean, for example, costs one).
In the notebook, observed tier counts of 330/290/180/100 against a claimed 40/30/20/10 split give χ² ≈ 5.1 (p ≈ 0.17), consistent with the claim. The test is the categorical analog of a one-sample test: it compares a whole distribution of counts to a benchmark. Always check that every expected count is at least about 5, or the approximation wobbles.
Test of Independence & Cramer's V
The test of independence works on a contingency table, rows for one categorical variable, columns for another, and asks whether they are related. The expected count for a cell, assuming independence, is (row total × column total) / grand total. Degrees of freedom = (rows − 1)(columns − 1).
A significant chi-square says the two variables are associated, the categorical cousin of correlation. But with enough data even a negligible association turns significant, so report Cramer's V, a rescaling of χ² to a 0-to-1 effect size. V near 0 means barely related; V near 1 means tightly linked. In the notebook a 2×3 table gives χ² ≈ 25 (p ≈ 0.000) with V ≈ 0.25, a moderate link.
Real-World Example: Support Channel by Region
A company logs each customer's region and preferred support channel (Email, Phone, Chat, App). Two questions, two chi-square tests: are the four channels used equally often (goodness-of-fit), and does channel preference depend on region (independence)?
One row per customer with region and the
preferred_channel (four categories).
| Question | Test | Result | Verdict |
|---|---|---|---|
| Channels used equally? | goodness-of-fit (df = 3) | χ² ≈ 13.7, p ≈ 0.003 | no, usage is uneven |
| Preference depends on region? | independence (df = 9) | χ² ≈ 81, p ≈ 10⁻¹³ | yes, associated |
| How strong is the link? | Cramer's V | ≈ 0.17 | modest, not dominant |
Both tests reject their nulls. The four channels are not used equally (goodness-of-fit p ≈ 0.003), and channel preference clearly depends on region (independence p ≈ 10⁻¹³). Yet the effect size tells the fuller story: Cramer's V ≈ 0.17 means the association, though unmistakably real given 900 customers, is modest, with the channel mix differing only somewhat from region to region. The test shows association, so it cannot say whether region drives channel choice or something else drives both. This is the discipline the chapter keeps returning to: a microscopic p-value confirms an effect exists, but only the effect size says whether it matters.
Chi-Square in Machine Learning & AI
The chi-square statistic shows up wherever categorical features and class labels meet.
| Idea (this chapter) | In ML / AI it becomes | Example |
|---|---|---|
| Test of independence | Categorical feature selection | chi2 in scikit-learn's SelectKBest |
| Observed vs expected | Detecting data / label drift | this month's category mix vs baseline |
| Goodness-of-fit | Checking class balance | do label counts match the design? |
| Cramer's V | Strength of categorical association | which features actually relate to the target |
Chi-square is a standard feature-selection tool for categorical predictors: scikit-learn's
chi2 score ranks count or one-hot features by a chi-square statistic built from each feature's
totals within each class. It is a close cousin of the test of independence, though it uses only the counts where
a feature is present, so its values differ from a full contingency-table test. It is also a workhorse for drift detection, comparing the
category distribution of incoming data against a training-time baseline flags when the world has shifted under
a deployed model. As always, pair the test with an effect size (Cramer's V): at web scale even trivial
associations come out "significant", so the size of the association is what guides which features and which drifts deserve action.
Run both chi-square tests in Python
The companion notebook computes the statistic by hand, runs goodness-of-fit with chisquare and
independence with chi2_contingency, derives Cramer's V, and loads
chi-square-tests--customer_channel.xlsx to test channel usage and its dependence on region.
View opens the rendered notebook instantly (no setup). Open in Colab runs &
edits it live in your browser. To run locally, install numpy, pandas, scipy,
matplotlib, statsmodels, and openpyxl and launch jupyter notebook.
🎓 Key Takeaways
- ✓Chi-square compares observed to expected counts: χ² = Σ(O − E)²/E, large when they diverge.
- ✓Goodness-of-fit tests one variable vs a claimed split (df = categories − 1).
- ✓Independence tests whether two categorical variables are related (df = (r−1)(c−1)); keep expected counts ≥ ~5.
- ✓Worked example: channels used unequally (p ≈ 0.003) and depend on region (p ≈ 10⁻¹³), but Cramer's V ≈ 0.17 says modestly.
- ✓In ML/AI: chi-square powers categorical feature selection and drift detection, paired with Cramer's V.
Practice Challenges
Five short challenges, beginner to intermediate. Try them before checking the solutions.
Goodness-of-fit for a die
Test whether [18, 22, 16, 20, 14, 10] (100 rolls) is consistent with a fair die.
Goodness-of-fit vs a claim
Observed [330, 290, 180, 100]; claimed split 40/30/20/10. Test it.
Test of independence
For the table [[90, 60, 50], [40, 80, 70]], test independence and report the degrees of freedom.
stats.chi2_contingency.Cramer's V
For the table above, compute Cramer's V as an effect size.
Channel by region
Load chi-square-tests--customer_channel.xlsx and test independence of region and preferred_channel.
pd.crosstab first.A fully-worked solutions notebook walks through all five challenges, each verified in code. Try them yourself first, then compare.
Quiz: Test Yourself
Eight quick questions on chi-square tests. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.
z, t, F, and chi-square all lean on distributional assumptions. Nonparametric Tests handles skewed, ordinal, or outlier-ridden data by testing ranks instead of raw values.