What Is Machine Learning? The three paradigms, in code¶
This notebook makes the taxonomy concrete. Using one realistic customer table we run a supervised model (predict a label), a regression (predict a number), an unsupervised model (find segments with no label), and a tiny reinforcement-learning agent (learn from reward), then contrast the statistics mindset (explain) with the machine-learning mindset (predict). Library-first with scikit-learn, statsmodels, and seaborn.
import numpy as np, pandas as pd, matplotlib.pyplot as plt
import seaborn as sns # seaborn = high-level statistical plots (heatmaps, pairplots, count/bar plots)
from matplotlib.colors import ListedColormap
EM="#4338ca"; DEEP="#3730a3"; LIGHT="#c7d2fe"; INK="#1a2138"; GRID="#e6e9f2"; RED="#ef4444"; AMBER="#d97706"; GREEN="#059669"; BLUE="#2563eb"; PUR="#9333ea"; GREY="#94a3b8"; SLATE="#475569"; ORG="#4338ca"; CYAN="#0891b2"
plt.rcParams.update({"figure.facecolor":"white","axes.facecolor":"white","figure.dpi":110,"font.size":11,
"axes.edgecolor":GRID,"axes.grid":True,"grid.color":GRID,"axes.axisbelow":True,"axes.spines.top":False,
"axes.spines.right":False,"axes.titlesize":12,"axes.titleweight":"bold","legend.frameon":False})
sns.set_style("whitegrid")
BASE="https://raw.githubusercontent.com/johnfisher-ai/Statistics-Data-Science-AI-Visual-Book/main/data/"
import warnings; warnings.filterwarnings('ignore')
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
from sklearn.linear_model import LogisticRegression, LinearRegression
from sklearn.cluster import KMeans
from sklearn.metrics import accuracy_score, confusion_matrix, ConfusionMatrixDisplay, r2_score
import statsmodels.api as sm
pd.set_option('display.max_columns', 30)
try: df = pd.read_excel('../../data/what-is-machine-learning--customers.xlsx', sheet_name='Data')
except FileNotFoundError: df = pd.read_excel(BASE + 'what-is-machine-learning--customers.xlsx', sheet_name='Data')
print('shape:', df.shape)
df.head()
shape: (600, 8)
| customer_id | age | income_k | tenure_months | num_products | monthly_spend | support_calls | churned | |
|---|---|---|---|---|---|---|---|---|
| 0 | C7000 | 43 | 102 | 70 | 4 | 75.22 | 1 | 0 |
| 1 | C7001 | 53 | 86 | 34 | 1 | 48.97 | 1 | 0 |
| 2 | C7002 | 24 | 38 | 6 | 1 | 30.25 | 3 | 0 |
| 3 | C7003 | 30 | 30 | 23 | 1 | 29.26 | 4 | 0 |
| 4 | C7004 | 19 | 44 | 8 | 1 | 51.81 | 3 | 0 |
What this shows. Each row is one customer; each column is a measurement. In ML vocabulary the input columns (age, income, tenure, and so on) are the features and the column we might want to predict, churned (1 = left, 0 = stayed), is the label or target. That single distinction, features versus label, splits machine learning into its main branches: if a label is present we can do supervised learning; if not, unsupervised.
feat = ['age','income_k','tenure_months','num_products','monthly_spend','support_calls']
X, y = df[feat], df['churned']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
clf = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000)).fit(X_train, y_train)
pred = clf.predict(X_test)
print(f'trained on {len(X_train)} customers, tested on {len(X_test)} unseen ones')
print(f'test accuracy = {accuracy_score(y_test, pred):.1%}')
print(f'baseline (always predict "stayed") = {(y_test == 0).mean():.1%}')
fig, ax = plt.subplots(figsize=(4.8,4.2))
ConfusionMatrixDisplay(confusion_matrix(y_test, pred), display_labels=['stayed','churned']).plot(ax=ax, cmap='Blues', colorbar=False)
ax.set_title('Churn classifier: predicted vs actual'); plt.tight_layout(); plt.show()
trained on 450 customers, tested on 150 unseen ones test accuracy = 77.3% baseline (always predict "stayed") = 71.3%
What the code does. train_test_split holds out 25% of customers as a test set, so we can measure performance on data the model never saw during training (the golden rule of ML). make_pipeline(StandardScaler(), LogisticRegression()) chains two steps: standardize the features, then fit a classifier. We fit on the training set and predict on the test set.
Reading the output. The model learned a mapping from features to the churn label and is right about three-quarters of the time on unseen customers. That needs a yardstick. Only 29% of customers churn, so a "model" that always answers stayed is already right 71.3% of the time on this test set, and the classifier adds about 6 points over it. The confusion matrix breaks the result down, and its bottom row shows the classifier catching 29 of the 43 churners in the test set. The diagonal cells are correct predictions (stayed-called-stayed, churned-called-churned), the off-diagonal cells are the two kinds of mistake. This is supervised classification: the target is a category.
Xr = df[['age','income_k','num_products','tenure_months']]; yr = df['monthly_spend']
Xr_tr, Xr_te, yr_tr, yr_te = train_test_split(Xr, yr, test_size=0.25, random_state=0)
reg = LinearRegression().fit(Xr_tr, yr_tr)
print(f'test R2 = {reg.score(Xr_te, yr_te):.3f}')
fig, ax = plt.subplots(figsize=(5.2,4.6))
ax.scatter(yr_te, reg.predict(Xr_te), s=16, color=EM, alpha=0.5)
lims=[yr_te.min(), yr_te.max()]; ax.plot(lims, lims, ls='--', color=INK)
ax.set(title='Predicting monthly spend (regression)', xlabel='actual spend', ylabel='predicted spend')
ax.xaxis.set_major_formatter(plt.FuncFormatter(lambda v,_: f'${v:.0f}')); ax.yaxis.set_major_formatter(plt.FuncFormatter(lambda v,_: f'${v:.0f}'))
plt.tight_layout(); plt.show()
test R2 = 0.825
What changed. Everything is the same supervised recipe (split, fit, evaluate) except the target is now a number (monthly_spend) instead of a category. That is the difference between the two kinds of supervised learning: classification predicts a class, regression predicts a quantity. The points hug the diagonal and the test R-squared says the model explains most of the variation in the bill, exactly the regression idea from Part XV, now framed as an ML task.
Xu = StandardScaler().fit_transform(df[['income_k','monthly_spend','num_products','tenure_months']])
km = KMeans(n_clusters=3, n_init=10, random_state=0).fit(Xu)
df['segment'] = km.labels_
print('customers per discovered segment:', np.bincount(km.labels_).tolist())
print(df.groupby('segment')[['income_k','monthly_spend','num_products','tenure_months']].mean().round(1))
# the label is used only now, after clustering, to compare the segments
print('churn rate by segment:', df.groupby('segment')['churned'].mean().round(3).tolist())
print('churners by segment:', df.groupby('segment')['churned'].sum().tolist(), 'of', int(df['churned'].sum()))
fig, ax = plt.subplots(figsize=(6.4,4.6))
sc = ax.scatter(df.income_k, df.monthly_spend, c=df.segment, cmap='viridis', s=18, alpha=0.7)
ax.set(title='K-means found 3 customer segments (no label used)', xlabel='income (k)', ylabel='monthly spend')
ax.yaxis.set_major_formatter(plt.FuncFormatter(lambda v,_: f'${v:.0f}')); plt.tight_layout(); plt.show()
customers per discovered segment: [269, 89, 242]
income_k monthly_spend num_products tenure_months
segment
0 39.6 43.5 1.5 15.0
1 101.2 89.6 4.5 50.3
2 73.3 61.0 2.4 39.3
churn rate by segment: [0.502, 0.034, 0.149]
churners by segment: [135, 3, 36] of 174
What the code does. Notice there is no y here, only features. KMeans(n_clusters=3) looks for three natural groups by putting customers who are close together (after standardizing) into the same cluster. The group means reveal the story: the algorithm recovered a budget, a standard, and a premium segment, low-to-high income and spend, entirely from the data's shape. The groups are broad, and in the scatter their edges overlap in income and spend. The churn label comes in only after the clustering, to compare the segments. The budget segment churns at 50%, the standard at 15% and the premium at 3%, and the budget segment holds 135 of the 174 churners. Unsupervised learning does not predict a known answer; it discovers structure (segments, topics, anomalies) when no label exists.
# 3 offers with unknown true response rates; the agent must LEARN which to send, from feedback alone
true_rates = [0.03, 0.12, 0.07] # hidden from the agent
rng = np.random.default_rng(7)
counts = np.zeros(3); values = np.zeros(3); history = []
for t in range(3000):
a = rng.integers(3) if rng.random() < 0.1 else int(np.argmax(values)) # explore 10%, else exploit best-so-far
reward = 1.0 if rng.random() < true_rates[a] else 0.0 # send offer, observe response
counts[a] += 1; values[a] += (reward - values[a]) / counts[a] # update the running estimate
history.append(reward)
print('learned value of each offer:', values.round(3), ' -> agent settled on offer', int(np.argmax(values))+1)
fig, ax = plt.subplots(figsize=(7.2,3.8))
ax.plot(np.arange(1,len(history)+1), np.cumsum(history)/np.arange(1,len(history)+1), color=EM, lw=2, label='learning agent')
ax.axhline(np.mean(true_rates), color=GREY, ls='--', label='random offer (average)')
ax.axhline(max(true_rates), color=GREEN, ls=':', label='best possible')
ax.set(title='Reinforcement learning: reward improves with experience', xlabel='offers sent', ylabel='average reward'); ax.legend()
plt.tight_layout(); plt.show()
learned value of each offer: [0.027 0.115 0.067] -> agent settled on offer 2
What this shows. There is no training table here at all. The agent acts (sends one of three offers), receives a reward (did the customer respond?), and updates its estimate of each offer's value, mostly picking the best one so far but occasionally exploring. Over a few thousand tries its average reward climbs from the random baseline toward the best-possible rate, and it correctly identifies offer 2 as the winner. That is reinforcement learning: learning a policy from trial and reward, the family behind game-playing and robotics agents.
# STATISTICS mindset: WHY do customers churn? fit for interpretation, read coefficients + p-values
Xc = sm.add_constant(StandardScaler().fit_transform(df[feat]))
logit = sm.Logit(df['churned'], Xc).fit(disp=0)
stat = pd.DataFrame({'coef': logit.params.values.round(3), 'p_value': logit.pvalues.values.round(3)}, index=['const']+feat)
print('STATISTICS (inference):'); print(stat.to_string()); print()
# MACHINE-LEARNING mindset: how well will it PREDICT unseen customers? cross-validate
cv = cross_val_score(make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000)), X, y, cv=5)
print(f'MACHINE LEARNING (prediction): 5-fold accuracy = {cv.mean():.1%} (+/- {cv.std():.1%})')
STATISTICS (inference):
coef p_value
const -1.416 0.000
age 0.020 0.890
income_k -0.789 0.000
tenure_months -1.180 0.000
num_products 0.160 0.471
monthly_spend 0.403 0.117
support_calls 0.468 0.000
MACHINE LEARNING (prediction): 5-fold accuracy = 79.3% (+/- 4.3%)
The contrast. Both use the same logistic model, but ask different questions. Statistics asks which factors are associated with churn, and are the associations significant?, so it reports coefficients and p-values. Short tenure, lower income and frequent support calls go with higher churn, each with a p-value that rounds to 0.000. The data are observational, so these coefficients describe associations and do not show that changing tenure or income would change churn. Machine learning asks how accurately will this predict customers I have not seen?, so it reports a cross-validated accuracy. Explanation versus generalization: the same math, two goals, and good practice borrows from both.
The taxonomy, in one view¶
- Supervised learning has a labeled target: classification predicts a category (churn yes/no), regression predicts a number (the monthly bill). It is by far the most common industry setting.
- Unsupervised learning has no label: it finds structure, clusters (customer segments), dimensionality reduction, anomaly detection.
- Reinforcement learning has no dataset at all: an agent learns a policy from reward by interacting with an environment.
- Statistics vs ML is a difference of emphasis, not tools: statistics leans on inference (explain, with uncertainty), ML leans on prediction (generalize to new data, measured by held-out performance).