This is a turning point in the book. The last three Parts gave you the moves; this Part puts them to work. Before the case studies begin, let us step back and assemble everything into a single, repeatable process, the one a working analyst runs on every new dataset.
Three chapters, three datasets, one routine. Each case study starts with a raw, deliberately messy file and walks start to finish: a first-contact audit, descriptive statistics, the right charts, then cleaning, handling duplicates, missing values, outliers, transformations, reshaping, and encoding, with a plain-English justification at each step. By the end you will have seen the entire first half of the book applied three times, on data that looks like what you meet in the wild.
The Story So Far
Everything we are about to use was built in Describing Data, Visualizing Data, and Preparing Data for Analysis. Three Parts, three jobs:
| Part | The job | Key tools you now own |
|---|---|---|
| II · Describing Data | Summarize a variable in numbers | Mean / median / mode, variance & SD, IQR, quartiles & percentiles, skewness & kurtosis, z-scores |
| III · Visualizing Data | See the shape and the relationships | Histograms, box plots, bar charts, scatter plots, correlation heatmaps; choosing the right chart |
| IV · Preparing Data | Make it clean and analysis-ready | Deduplication, missing-data handling, outlier treatment, transformations, reshaping/joins, encoding & scaling, EDA |
Notice the arc: you cannot prepare data well until you have described and visualized it, because the summaries and the plots are what reveal the duplicates, the skew, the outliers, and the gaps. Describing, visualizing, and preparing are not three separate phases done once each; they interleave, every cleaning step sends you back to re-describe and re-plot.
The EDA Workflow, One Picture
Here is the loop every case study follows. Raw data goes in; an analysis-ready table comes out; and the messy middle is the Preparing Data for Analysis toolbox, applied in roughly this order but revisited as often as needed.
The order above is a sensible default, not a rigid script. Imputing a column changes its distribution, so you re-plot it. Removing an outlier shifts the mean, so you re-describe. The dashed arc is the whole point: exploratory data analysis circles back on itself until the data is trustworthy. Each case study makes that looping explicit.
Meet the Three Datasets
We will work three datasets, chosen because they are intuitive, mix numeric and categorical columns, and carry
exactly the kinds of mess you learn the most from. Each ships as a real comma-separated file in the book's
data/ folder.
The mess is intentional and curated: together these three files cover nearly every problem from Preparing Data for Analysis, so by the end of the Part you will have treated all of them at least once. The companion notebook runs the same first-contact audit on all three so you can see the issues for yourself before any cleaning begins.
How to Read a Case Study
Every case-study chapter follows the same five beats, so once you learn the rhythm you can focus on the decisions rather than the structure:
| Beat | What happens | The question answered |
|---|---|---|
| 1 · First contact | Load the file, audit shape, dtypes, duplicates, missingness | "What am I even looking at?" |
| 2 · Describe | Central tendency, spread, and shape of the key variables | "Where does each variable sit, and how messy is it?" |
| 3 · Visualize | Histograms, box plots, scatters, a correlation heatmap | "What do the distributions and relationships look like?" |
| 4 · Prepare | Dedup, fix missing, treat outliers, transform, reshape, encode | "What has to change, and why exactly this fix?" |
| 5 · Recap | Before/after summary and the analysis-ready table | "What did we decide, and what did it buy us?" |
Cleaning is full of judgment calls, drop or impute, remove an outlier or keep it, log-transform or not. The difference between a beginner and a professional is not which choice they make; it is that the professional writes down why. Each case study states the reason for every step, because a cleaning decision you cannot justify is one you cannot defend, reproduce, or trust later.
Surveys of working data scientists repeatedly find that the majority of their time goes to exactly this: finding, cleaning, and organizing data. Models and algorithms get the headlines, but the quiet, careful work in these three chapters is what most of the job actually looks like, and it is what separates an analysis you can stake a decision on from one you cannot.
Bring it to life in Python
The companion notebook loads all three case-study CSVs and runs one reusable first-contact audit on each: shape, dtypes, duplicate rows, and missingness, then previews the specific mess in every file (Spotify's clashing scales, Ames's 99%-missing column and mixed date formats, the penguins' inconsistent sex column and impossible flipper value). It is your before picture for the three chapters ahead.
View opens the rendered notebook instantly (no setup). Open in Colab runs &
edits it live in your browser. To run locally, install numpy and pandas and launch
jupyter notebook.
🎓 Key Takeaways
- ✓This Part is a working review: Describing, Visualizing, and Preparing Data applied end to end on three real, messy datasets.
- ✓One routine: raw data → describe → visualize → prepare → analysis-ready, looping back as every fix changes the data.
- ✓The Preparing Data for Analysis toolbox, dedup, missing data, outliers, transforms, reshape, encode/scale, is applied as the data needs it, not in lockstep.
- ✓The three datasets (Spotify, Ames, Penguins) are curated to be messy so they cover nearly every cleaning technique between them.
- ✓The professional habit: log the reason for every cleaning decision, a fix you cannot justify is one you cannot trust.
Practice Challenges
Five short challenges to warm up before the case studies. Try them on the three datasets in Python before checking the solutions.
A reusable health check
Write one function that, for any DataFrame, returns its row count, duplicate-row count, and the three most-missing columns. Run it on the Spotify data.
df.duplicated().sum() and df.isna().mean().sort_values().Match the tool to the symptom
For Spotify, quantify three problems: how many tempo values are an impossible 0, how different
the feature scales are, and how skewed popularity is. Name the fix for each.
Drop or impute?
List every Ames column with missing values and its missing percentage. State a rule for dropping a column versus imputing it, and apply it. Which column is the obvious drop, and why?
Clean a category
The penguins sex column has blanks, a stray "." and inconsistent casing. Standardize it to
male/female/NaN, drop rows with no measurements at all, and report how many complete rows remain.
.str.strip().str.lower(), map "." and "" to NaN, then dropna.Map the toolbox
Across all three datasets, match each problem you have seen (duplicates, missing values, outliers, skew, mixed date formats, scaling, encoding) to the Preparing Data for Analysis chapter that handles it.
A fully-worked solutions notebook answers all five on the real data, in the same visual style. Try them yourself first, then compare.
Quiz: Test Yourself
Eight questions reviewing Describing, Visualizing, and Preparing Data. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.