Everything in this book so far has been a piece of a puzzle: describing data, visualizing it, cleaning it, the logic of a test, the family of tests, effect sizes, correlation. A capstone puts the pieces together on one real problem, start to finish. The hard part is rarely the arithmetic, which a single line of Python now handles. The hard part is judgment: what kind of data is this, what is actually being asked, do the assumptions hold, and if they do not, what then? This chapter is the shared playbook for all of it.
Let the data type and design narrow the choice, let the assumptions settle it, run the test, and finish with an effect size, a confidence interval, and the ethics, never a bare p-value. Every capstone ahead follows exactly this arc, out loud.
The Twelve-Step Framework
The same twelve steps, grouped into four phases, run through every project. The first phase decides what you are studying, the middle two get the data honest and pick a valid test, and the last one turns a number into a decision. Skipping a step is where analyses go wrong: choosing a test before checking its assumptions, or reporting a p-value with no sense of how big the effect is.
| # | Step | What you actually do | Tools you will use |
|---|---|---|---|
| 1 | Goal & hypotheses | State the question in plain words, then as H₀ and H₁. Pick α up front. | a sentence, not code |
| 2 | Data types | Label every variable continuous, ordinal, or nominal. This alone rules most tests in or out. | df.dtypes, judgment |
| 3 | Describe | Counts, means, medians, spread, group sizes. Meet the dataset before touching it. | describe(), value_counts() |
| 4 | Visualize | Histograms, boxplots, violins, bars, scatter, Q-Q. See the shape, the groups, the outliers. | matplotlib, seaborn |
| 5 | Clean & prepare | Handle missing values, duplicates, impossible entries, and outliers. Transform if needed. | pandas, np.log |
| 6 | Design | One sample, independent groups, paired, factorial, or repeated? Name the structure. | a diagram, a sentence |
| 7 | Check assumptions | Normality (Shapiro-Wilk, K-S, Q-Q) and equal variance (Levene, Bartlett). Report each. | scipy.stats, pingouin |
| 8 | Decide & fix | Assumptions hold, use the parametric test. They fail, transform or switch to a rank-based test. | the fork in Section 3 |
| 9 | Choose the test | Data type, groups, and design point to exactly one test (with its nonparametric twin). | the map in Section 4 |
| 10 | Run & report | Compute the statistic, the p-value, the effect size, and the interval. Post-hoc if needed. | scipy, statsmodels |
| 11 | Interpret | Answer the original question in plain language. Separate statistical from practical significance. | plain English |
| 12 | Ethics & bias | Who was sampled, what could bias it, how far does the conclusion generalize, what is the risk of misuse? | honest reflection |
It All Starts With the Data Type
Before any test, ask what kind of number you have. This single question, from Levels of Measurement, does most of the work of choosing a test, because each data type admits a different family of methods.
| Data type | What it is | Examples | Typical methods |
|---|---|---|---|
| Continuous | Numbers you can average; equal intervals, a true or arbitrary zero. | weight, blood pressure, time, score | t-tests, ANOVA, Pearson correlation |
| Ordinal | Ordered categories without equal spacing; ranks and Likert scales. | satisfaction 1–7, pain 0–10, rankings | rank-based tests, Spearman, Kendall |
| Nominal | Unordered categories; labels and counts. | region, commute mode, pass/fail | chi-square, proportion tests |
Treating an ordinal scale as if it were continuous is the most common error in applied statistics. Averaging a 1-to-5 satisfaction score is tempting, but the gap from "poor" to "fair" need not equal the gap from "good" to "excellent." When a variable is ordinal or clearly non-normal, the rank-based tests (Capstones 14–16) are the honest choice, and this framework tells you exactly when to reach for them.
Assumptions Come First, Then the Test
The parametric tests, t-tests, ANOVA, and Pearson correlation, earn their power by assuming things about the data: roughly normal distributions and, when comparing groups, roughly equal variances. The discipline of every capstone is to check those assumptions before choosing the test, not after. If they hold, use the parametric test. If they fail, transform the data or switch to the rank-based twin, which makes no such assumption.
| Assumption | How to check it | What the test says |
|---|---|---|
| Normality | Shapiro-Wilk (best for small n), Kolmogorov-Smirnov, and a Q-Q plot by eye | a small p-value means "not normal"; trust the plot as much as the p |
| Equal variance | Levene's test (robust, preferred) or Bartlett's test (assumes normality) | a small p-value means the groups' spreads differ |
| Independence | from the study design, not a test: were observations gathered separately? | if paired or repeated, use the matched-design test |
| Enough data | expected counts ≥ ~5 (chi-square); ~10 successes and failures (proportions) | if too sparse, use an exact test |
Notice the order. The assumption tests do not choose the test for you; they tell you which version of the test is trustworthy. A skewed outcome with heavy outliers is not a reason to abandon the analysis, it is a signal to reach for the rank-based twin, which answers the same question without the normality assumption.
Choosing the Test
With the data type known and the assumptions checked, the test almost picks itself. Answer three questions, is the outcome numeric or categorical, how many groups, and are they independent or matched, and the map below lands you on a single method, with its rank-based fallback in parentheses for when the assumptions do not hold.
| Question | Design | Parametric test | If assumptions fail | Effect size |
|---|---|---|---|---|
| Mean vs a target | one sample | one-sample t-test | Wilcoxon signed-rank | Cohen's d |
| Two groups differ? | independent | two-sample (Welch) t-test | Mann-Whitney U | Cohen's d, rank-biserial |
| Change within subjects | paired | paired t-test | Wilcoxon signed-rank | Cohen's dz |
| 3+ groups differ? | independent | one-way ANOVA + Tukey | Kruskal-Wallis + Dunn | η², ε² |
| 3+ conditions, same subjects | repeated | repeated-measures ANOVA | Friedman | partial η² |
| Two factors + interaction | factorial | two-way ANOVA | (align-and-rank / robust) | partial η² |
| Two categories associated? | — | chi-square of independence | Fisher's exact | Cramer's V |
| Fits an expected split? | — | chi-square goodness-of-fit | exact multinomial | Cramer's V, w |
| A rate vs a target / another | — | one- / two-proportion z-test | exact binomial | risk difference, h |
| Two numeric variables move together? | — | Pearson correlation | Spearman ρ / Kendall τ | r itself, r² |
Report Honestly, and Fairly
A test that stops at "p < 0.05" is only a third of an answer. With a big enough sample almost anything is "significant," and with too small a sample a real effect hides. Every capstone closes the same way: three numbers together, then a plain-language verdict, then a look at what could make it wrong.
| Always report | The question it answers |
|---|---|
| p-value | Could chance alone produce a result this extreme? (yes/no at α) |
| Effect size | How big is the effect? (Cohen's d, η², Cramer's V, r) — the part that actually matters |
| Confidence interval | How precisely do we know it? A range, not a single point |
Then the twelfth step: ethics and bias. Every dataset was collected by someone, from someone, for some purpose, and those choices shape what the numbers can honestly say. A convenience sample does not represent a population. A survey question can lead its respondent. A significant result on a biased sample is a confident wrong answer. Two of the capstones ahead are built on real survey data precisely so we can walk through the questionnaire and the sampling method and ask, out loud, who this conclusion is really about.
Each capstone ships two artifacts, and the split is deliberate. A notebook does the reproducible work: it loads the data, runs the twelve steps, and produces every figure and number. A written report, authored by a statistician for a non-statistician, tells the story: what we found, what we did about the messy parts, why we chose the test we did, and what it means for the decision at hand. The computer makes the evidence; a person makes the case.
The Sixteen Capstones Ahead
Sixteen projects, grouped by the question they answer, each one a complete pass through the framework on its own downloadable dataset, with a full notebook and a statistician's written report. They become clickable here as each one is published; the Contents always shows what is live.
🎓 Key Takeaways
- ✓One framework, twelve steps: frame the question, explore and prepare the data, validate and choose, then test and conclude. Every capstone follows it.
- ✓Data type first: continuous, ordinal, or nominal decides the family of tests before anything else.
- ✓Check assumptions, then choose: Shapiro-Wilk and K-S for normality, Levene and Bartlett for equal variance, and switch to a rank-based test when they fail.
- ✓Three questions pick the test: outcome type, number of groups, and independent vs matched, each with a nonparametric twin.
- ✓Finish honestly: a p-value, an effect size, and an interval together, plus a candid look at sampling, bias, and generalizability.
Quiz: Test Yourself
Eight quick questions on the framework the whole part is built on. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.