🎯 What you'll build in this notebook¶
| # | Demo | Concept |
|---|---|---|
| 1 | Classify the columns of a dataset | Qualitative vs quantitative |
| 2 | Pick the right chart for each type | Bar chart vs histogram |
| 3 | Counts vs measurements | Discrete vs continuous |
| 4 | Cause, effect & a confounder | Variable roles (IV / DV / confounder) |
| 5 | Binning a continuous variable | Transforming continuous → categorical |
⚙️ Setup, imports & the book's plotting style¶
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
rng = np.random.default_rng(11)
NAVY="#0a1230"; INK="#1a2138"; INK_SOFT="#4a5578"
CYAN="#0891b2"; PURPLE="#7c3aed"; AMBER="#d97706"
GREEN="#059669"; PINK="#db2777"; BLUE="#2563eb"; GRID="#e6e9f2"
plt.rcParams.update({
"figure.facecolor":"white","axes.facecolor":"white","figure.dpi":110,"font.size":11,
"axes.edgecolor":GRID,"axes.linewidth":1.2,"axes.grid":True,"grid.color":GRID,"axes.axisbelow":True,
"axes.spines.top":False,"axes.spines.right":False,"axes.titlesize":15,"axes.titleweight":"bold","axes.titlecolor":INK,
"axes.labelcolor":INK_SOFT,"axes.labelsize":11.5,"xtick.color":INK_SOFT,"ytick.color":INK_SOFT,"legend.frameon":False,
})
def titlecard(ax, title, subtitle=None):
ax.set_title(title, loc="left", pad=18)
if subtitle:
ax.text(0, 1.02, subtitle, transform=ax.transAxes, fontsize=10.5, color=INK_SOFT, va="bottom")
print("✅ Environment ready.")
✅ Environment ready.
people = pd.DataFrame({
"name": ["Ana","Ben","Cara","Dan","Eve","Finn"],
"gender": ["F","M","F","M","F","M"],
"satisfaction": ["High","Low","Medium","High","Medium","Low"],
"num_pets": [0, 2, 1, 3, 0, 1],
"height_cm": [166.2, 178.5, 160.0, 182.3, 171.1, 175.9],
})
print(people)
print("\nWhat pandas sees (dtypes):")
print(people.dtypes)
name gender satisfaction num_pets height_cm 0 Ana F High 0 166.2 1 Ben M Low 2 178.5 2 Cara F Medium 1 160.0 3 Dan M High 3 182.3 4 Eve F Medium 0 171.1 5 Finn M Low 1 175.9 What pandas sees (dtypes): name str gender str satisfaction str num_pets int64 height_cm float64 dtype: object
# A human-readable classification of each column
classification = pd.DataFrame({
"column": ["gender", "satisfaction", "num_pets", "height_cm"],
"big type":["Qualitative", "Qualitative", "Quantitative", "Quantitative"],
"subtype": ["Nominal (no order)", "Ordinal (has order)", "Discrete (counts)", "Continuous (measures)"],
"example value":["F", "High", "2", "178.5"],
})
classification
| column | big type | subtype | example value | |
|---|---|---|---|---|
| 0 | gender | Qualitative | Nominal (no order) | F |
| 1 | satisfaction | Qualitative | Ordinal (has order) | High |
| 2 | num_pets | Quantitative | Discrete (counts) | 2 |
| 3 | height_cm | Quantitative | Continuous (measures) | 178.5 |
Key idea: qualitative (categorical) data describe qualities (gender, satisfaction); quantitative (numerical) data are numbers you can do math on (pets, height). Note that satisfaction looks like text but has a natural order, that's ordinal.
fig, (a1, a2) = plt.subplots(1, 2, figsize=(11,4.4))
# Categorical -> bar chart (count per category)
order = ["Low","Medium","High"]
counts = people["satisfaction"].value_counts().reindex(order)
a1.bar(order, counts.values, color=CYAN, alpha=0.9, edgecolor="white", width=0.6)
titlecard(a1, "Categorical → Bar Chart", "Satisfaction levels · bars have GAPS")
a1.set_ylabel("People"); a1.set_yticks(range(0,4))
# Continuous -> histogram (bins touch)
heights = rng.normal(172, 8, 300) # a bigger sample to show the shape
a2.hist(heights, bins=15, color=PURPLE, alpha=0.85, edgecolor="white")
titlecard(a2, "Continuous → Histogram", "Heights · bins TOUCH (a range each)")
a2.set_xlabel("Height (cm)"); a2.set_ylabel("People")
plt.tight_layout(); plt.show()
Spot the difference: a bar chart compares separate categories (gaps between bars); a histogram shows the distribution of a continuous number (bars touch because each covers a range).
children = rng.poisson(2, 400) # discrete: 0,1,2,3,... (you can count them)
temps = rng.normal(21, 4, 400) # continuous: 21.3, 18.97, ... (you measure them)
fig, (a1, a2) = plt.subplots(1, 2, figsize=(11,4.4))
vals, cnts = np.unique(children, return_counts=True)
a1.bar(vals, cnts, color=AMBER, alpha=0.9, edgecolor="white", width=0.6)
titlecard(a1, "Discrete: children per family", "Only whole numbers — distinct bars")
a1.set_xlabel("Number of children"); a1.set_ylabel("Families"); a1.set_xticks(vals)
a2.hist(temps, bins=20, color=GREEN, alpha=0.85, edgecolor="white")
titlecard(a2, "Continuous: room temperature", "Any value in a range — smooth shape")
a2.set_xlabel("Temperature (°C)"); a2.set_ylabel("Readings")
plt.tight_layout(); plt.show()
print("Quick test: can you list every possible value?")
print(" children -> 0,1,2,3,... YES -> DISCRETE")
print(" temperature -> 21.0, 21.01, 21.001, ... NO -> CONTINUOUS")
Quick test: can you list every possible value? children -> 0,1,2,3,... YES -> DISCRETE temperature -> 21.0, 21.01, 21.001, ... NO -> CONTINUOUS
# Classic example: do ice-cream sales CAUSE drownings? (They don't!)
temperature = rng.uniform(10, 35, 200) # the hidden CONFOUNDER
ice_cream = 20 + 6*temperature + rng.normal(0,15,200) # driven by temperature
drownings = 1 + 0.30*temperature + rng.normal(0,1.2,200) # ALSO driven by temperature
corr_fake = np.corrcoef(ice_cream, drownings)[0,1]
print(f"Correlation(ice-cream sales, drownings) = {corr_fake:.2f} <- looks strong!")
print("But neither causes the other — hot weather (temperature) drives BOTH.")
Correlation(ice-cream sales, drownings) = 0.82 <- looks strong! But neither causes the other — hot weather (temperature) drives BOTH.
fig, (a1, a2) = plt.subplots(1, 2, figsize=(11,4.6))
# Left: the spurious relationship
a1.scatter(ice_cream, drownings, color=PINK, s=35, alpha=0.7, edgecolor="white")
titlecard(a1, "The Trap: a spurious correlation", f"ice cream vs drownings · r = {corr_fake:.2f}")
a1.set_xlabel("Ice-cream sales"); a1.set_ylabel("Drownings")
# Right: reveal the confounder driving both
a2.scatter(temperature, ice_cream/ice_cream.max(), color=AMBER, s=28, alpha=0.7, label="ice-cream (scaled)")
a2.scatter(temperature, drownings/drownings.max(), color=BLUE, s=28, alpha=0.7, label="drownings (scaled)")
titlecard(a2, "The Truth: temperature drives both", "the confounder revealed")
a2.set_xlabel("Temperature (°C)"); a2.set_ylabel("scaled value"); a2.legend(fontsize=9)
plt.tight_layout(); plt.show()
Variable roles in one line: IV = suspected cause, DV = measured effect, confounder = a hidden third variable that influences both. Spotting confounders is why "correlation ≠ causation", a theme we revisit in the Correlation chapter.
# A continuous variable: ages
ages = rng.normal(38, 12, 300).clip(18, 80)
# Bin it into ordered categories (continuous -> ordinal)
bins = [18, 30, 45, 60, 81]
labels = ["18-29", "30-44", "45-59", "60+"]
age_group = pd.cut(ages, bins=bins, labels=labels, right=False)
counts = age_group.value_counts().reindex(labels)
print("Binning turns a continuous variable into ordinal categories:")
print(counts.to_string())
fig, ax = plt.subplots(figsize=(8.5,4.2))
ax.bar(labels, counts.values, color=PURPLE, alpha=0.85, edgecolor="white")
titlecard(ax, "Binning a continuous variable", "ages (continuous) grouped into ordered age brackets (ordinal)")
ax.set_xlabel("Age group"); ax.set_ylabel("People")
plt.tight_layout(); plt.show()
Binning turns a continuous variable into ordinal categories: 18-29 53 30-44 153 45-59 80 60+ 14
Takeaway: binning trades detail for simplicity. Ages (continuous) become age brackets (ordinal), handy for charts and some models, but you lose the exact values, so bin on purpose, not by habit.
- Qualitative (nominal/ordinal) describes qualities; quantitative (discrete/continuous) is numbers.
- Bar charts are for categories (gaps); histograms are for continuous data (touching bins).
- Discrete = countable whole numbers; continuous = any value in a range.
- Variable roles: independent (cause) → dependent (effect), with confounders lurking in between.
- Data type decides everything downstream, the chart, the statistic, and the model you may use.