Resolution times, incomes, and wait times are rarely bell-shaped, they pile up near zero with a long tail of slow cases. The mean is then a poor summary and the t-test rests on shaky ground. Nonparametric tests sidestep the problem by ranking the data and testing the ranks.
Rank the data, then test the ranks. Mann-Whitney replaces the two-sample t-test, Kruskal-Wallis replaces one-way ANOVA, and Wilcoxon signed-rank replaces the paired t-test.
When Parametric Tests Break
The z, t, and F tests assume roughly normal data and revolve around means, which a single outlier or a long tail can distort badly. (Chi-square is the exception among the classic tests: it works on counts, and asks for adequate expected cell frequencies rather than normality.) Rank-based tests inherit the median's robustness: replacing values with ranks caps the influence of any one extreme point.
In the notebook, one fat-fingered outlier shifts the mean from 50 to 59 but leaves the median essentially unchanged. Any test built on the mean inherits that fragility; a rank-based test treats the outlier as merely "the largest value" and is barely moved. Reach for nonparametric methods when data are skewed, heavy-tailed, or ordinal.
Mann-Whitney U
The Mann-Whitney U test (also called the Wilcoxon rank-sum test) is the nonparametric answer to the two-sample t-test. Pool both groups, rank everything, and ask whether one group's values tend to outrank the other's, no normality required.
U counts how often a value from one group exceeds a value from the other. In the notebook, two skewed groups give a clean Mann-Whitney verdict where a t-test would be on thin ice. The test detects a shift between the distributions, typically reported alongside the two medians.
Kruskal-Wallis & Wilcoxon
Each parametric test has a rank-based twin. Kruskal-Wallis extends Mann-Whitney to three or more groups (the nonparametric one-way ANOVA). Wilcoxon signed-rank is the paired version, ranking the within-pair differences in place of the paired t-test.
| Parametric test | Nonparametric twin | Use when |
|---|---|---|
| Two-sample t-test | Mann-Whitney U (rank-sum) | two independent groups, skewed/ordinal |
| Paired t-test | Wilcoxon signed-rank | paired data, skewed differences |
| One-way ANOVA | Kruskal-Wallis | 3+ groups, non-normal |
| Pearson correlation | Spearman correlation | monotonic but non-linear / ordinal |
The trade-off is power: when the data really are normal, the parametric test is a little more sensitive. But when assumptions fail, the rank-based test is both valid and only modestly less powerful, a safe default for messy real-world data.
Real-World Example: Ticket Resolution Times
A support organization logs how long each ticket takes to resolve. The distribution is heavily right-skewed, most tickets close quickly, a few drag on, so the mean misleads and the t-test is shaky. We compare two teams (Mann-Whitney) and three priority levels (Kruskal-Wallis) on ranks.
One row per ticket with team, priority,
resolution_hours (strongly right-skewed), and a 1-5 csat rating.
| Question | Test | Result | Verdict |
|---|---|---|---|
| Alpha vs Bravo speed? | Mann-Whitney U | medians 9.6 vs 7.8 h, p ≈ 0.047 | Bravo faster |
| Differ by priority? | Kruskal-Wallis | H ≈ 29.6, p ≈ 10⁻⁷ | yes, strongly |
| Shape of the data | skewness | ≈ 2.5 (long right tail) | ranks safer than means |
Both rank tests are decisive. Bravo resolves tickets faster than Alpha by median (7.8 vs 9.6 h; Mann-Whitney p ≈ 0.047), and resolution time differs sharply across priority levels (Kruskal-Wallis H ≈ 29.6, p ≈ 10⁻⁷), with High-priority tickets clearing fastest. Because the data are so right-skewed (skewness ≈ 2.5), these rank-based conclusions are far more trustworthy than a mean-based t-test or ANOVA, which the long tail would distort.
Nonparametric Methods in Machine Learning & AI
Ranks are deeper in ML than they look, including the most common classifier metric of all.
| Idea (this chapter) | In ML / AI it becomes | Example |
|---|---|---|
| Mann-Whitney U | The AUC of a classifier | ROC-AUC equals the normalized U statistic |
| Rank-based comparison | Robust model evaluation | compare skewed latency / loss distributions |
| Spearman correlation | Monotonic feature relationships | rank-correlation feature screening |
| Wilcoxon signed-rank | Paired model comparison (non-normal) | per-example score differences, robustly |
The single most reported classifier metric, ROC-AUC, is exactly a rescaled Mann-Whitney U statistic: it is the probability that a random positive example is scored above a random negative one, a pure rank comparison. That is why AUC is invariant to any monotonic rescaling of the scores. Rank-based tests also give robust model comparisons when metrics like latency or per-example loss are heavy-tailed, and Spearman correlation is the go-to when a relationship is monotonic but not linear. When in doubt about distributional assumptions, ranks are the safe currency.
Run the rank-based tests in Python
The companion notebook shows how one outlier wrecks the mean but not the ranks, runs Mann-Whitney,
Kruskal-Wallis, and Wilcoxon signed-rank, and loads nonparametric-tests--ticket_times.xlsx to compare teams and
priorities on heavily skewed resolution times.
View opens the rendered notebook instantly (no setup). Open in Colab runs &
edits it live in your browser. To run locally, install numpy, pandas, scipy,
matplotlib, statsmodels, and openpyxl and launch jupyter notebook.
🎓 Key Takeaways
- ✓Nonparametric tests work on ranks, so they tolerate skew, outliers, and ordinal data.
- ✓Mann-Whitney U → two-sample t; Kruskal-Wallis → one-way ANOVA; Wilcoxon signed-rank → paired t.
- ✓Trade-off: slightly less power than a parametric test when normality truly holds, but valid when it does not.
- ✓Real data: on skewed ticket times, Bravo beats Alpha (p ≈ 0.047) and priorities differ (p ≈ 10⁻⁷).
- ✓In ML/AI: ROC-AUC is a rescaled Mann-Whitney U; ranks power robust evaluation and Spearman correlation.
Practice Challenges
Five short challenges, beginner to intermediate. Try them before checking the solutions.
Mann-Whitney U
Compare lognormal(1.0, 0.6, n = 50) and lognormal(0.6, 0.6, n = 50) with the rank-sum test.
stats.mannwhitneyu.Outlier robustness
Add an outlier of 500 to a clean N(50, 8, 40) sample; compare how the t-test and Mann-Whitney p-values react.
Kruskal-Wallis
Compare three skewed groups (lognormal μ = 1.0, 1.2, 1.5, n = 40 each).
stats.kruskal(g1, g2, g3).Wilcoxon signed-rank
Paired before/after where after = before × Uniform(0.7, 0.95). Test the paired drop.
stats.wilcoxon(after, before).Real data: ticket times
Load nonparametric-tests--ticket_times.xlsx; run Mann-Whitney (Alpha vs Bravo) and Kruskal-Wallis (by priority).
resolution_hours appropriately.A fully-worked solutions notebook walks through all five challenges, each verified in code. Try them yourself first, then compare.
Quiz: Test Yourself
Eight quick questions on nonparametric tests. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.
You now have a whole toolbox: z, t, ANOVA, chi-square, and the rank-based tests. Choosing the Right Test turns it into a decision map, match the question and data type to the test.