Case Study: Spotify Track Features

Time to put the whole toolkit to work. This chapter follows the five-beat routine from the Review & How to Read These Case Studies chapter, first contact, describe, visualize, prepare, recap, on a dataset of Spotify track features. The data is fun and intuitive, and it is messy in exactly the ways that teach the most.

Before describing or cleaning anything, we look. The first-contact audit (shape, dtypes, duplicates, missingness, and how the categories were actually typed) turns a mystery file into a concrete to-do list.

With the problems cataloged, we explore. Two things jump out immediately, one about the scales of the features, and one about how they relate.

Every feature lives on a different scale

Descriptive statistics show the audio features do not share a number range at all: danceability is a decimal near 1, loudness is negative decibels, tempo runs to the hundreds, and duration_ms is in the hundreds of thousands. On one shared (logarithmic) axis the gap is impossible to miss:

One strong relationship, and a surprise

A correlation heatmap of the seven numeric features is almost entirely pale, the features are mostly independent, except for one hot cell: energy and loudness correlate at r ≈ 0.94. The surprise is the popularity row: it is near-zero against every audio feature, what makes a song popular here is not its sound.

Now we fix the to-do list, in order, and we say why at each step.

Duplicates first (see Finding & Removing Duplicates and Inconsistencies). Eight rows are exact copies of other rows. Left in, they would double-count those songs in every average and every chart, so drop_duplicates() removes them: 915 → 907 rows.

Standardize the categories (see Finding & Removing Duplicates and Inconsistencies). The genre column was typed nineteen ways ("Pop", "pop ", "POP", "EDM" for electronic, and so on). Until those collapse to the eight real genres, any group-by treats them as different. Trimming whitespace and mapping the variants fixes it; mode shrinks from eight spellings to two (Major / Minor) the same way.

Missing values, each on its merits (see Handling Missing Data). This is the step that rewards thought: the same problem has three different right answers depending on the column.

Treat the outliers (see Detecting & Treating Outliers). A box plot of duration_ms shows a few tracks running to impossible lengths, one at 18,000,000 ms, five hours. The IQR rule flags a couple dozen long songs, but most are genuine (extended mixes, classical works). Only three are physically impossible, so we remove just those errors and keep the real long songs. An outlier rule detects; a human decides.

Transform the skew (see Transformations). Even after the errors are gone, duration has a long right tail (skew ≈ +0.93). A natural-log transform straightens it to near-symmetric (skew ≈ +0.04), which steadies any model that prefers well-behaved inputs.

Scale, then encode by type (see Feature Engineering). Recall the log-axis chart: the features are on utterly different scales. Standardizing rescales each to mean 0 and SD 1 so none dominates by sheer magnitude. Then the categories are encoded according to whether they have an order. The file carries two extra category columns we cleaned alongside genre: audio_quality (low / standard / high) and mood (happy / sad / chill), both typed with messy casing. The first is ordinal, so it becomes integer codes that keep the order; the second is nominal, so it becomes one-hot columns, exactly like genre.

Here is the whole journey, from raw file to analysis-ready table:

Cleaning was never the goal; understanding is. Now that the duplicates, missing values, and errors are gone, the describing toolkit from Describing Data and the charts from Visualizing Data finally tell the truth. This is the payoff beat: a last, exploratory look at what the data actually says.

Measurements: the numbers behind two features

The full descriptive summary of tempo and popularity, computed on the clean data, pulls together everything from Chapters 8 to 11:

The standout is the coefficient of variation: at 58% versus 23%, songs differ far more in popularity than in tempo once you account for their different averages, the kind of comparison a raw standard deviation cannot make across two different units (see Measures of Dispersion). And tempo's high kurtosis says most songs huddle around 118 BPM with a few far-off tails, a shape no single average reveals.

A bar chart for the categories (see Charts for Categorical Data)

With genres finally standardized, a simple bar chart of counts is trustworthy: the eight genres are reasonably balanced, with pop, rock, and hip-hop the most common.

Grouped box plots reveal hidden structure (Chapters 15 & 16)

The most useful exploratory chart compares a numeric variable across a category. Box plots of energy by genre split cleanly into two clusters, an upbeat group (pop, rock, electronic) and a mellow group (jazz, country, r&b, hip-hop, classical), structure that was not safe to read until the data was clean.

The last visual check: all the prep work, at a glance

Before declaring victory, one final summary. Every problem the audit found, and what we did about it, on a single scorecard:

View opens the rendered notebook instantly (no setup). Open in Colab runs & edits it live in your browser. To run locally, install numpy, pandas, scikit-learn, scipy, matplotlib and launch jupyter notebook.

Five challenges on the Spotify data, following the case study beat by beat. Try them in Python before checking the solutions.

Eight questions on the Spotify case study. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.