Chapter 111 · Solutions
Core Classification & Regression Algorithms · Solutions
Five challenges, each verified in code.
Chapter 111 · Practice Challenges, Solved¶
Five exercises on the core algorithms, worked in scikit-learn on the Chapter 111 heart-disease table.
In [1]:
import numpy as np, pandas as pd, matplotlib.pyplot as plt
import seaborn as sns # seaborn = high-level statistical plots (heatmaps, pairplots, count/bar plots)
from matplotlib.colors import ListedColormap
EM="#4338ca"; DEEP="#3730a3"; LIGHT="#c7d2fe"; INK="#1a2138"; GRID="#e6e9f2"; RED="#ef4444"; AMBER="#d97706"; GREEN="#059669"; BLUE="#2563eb"; PUR="#9333ea"; GREY="#94a3b8"; SLATE="#475569"; ORG="#4338ca"; CYAN="#0891b2"
plt.rcParams.update({"figure.facecolor":"white","axes.facecolor":"white","figure.dpi":110,"font.size":11,
"axes.edgecolor":GRID,"axes.grid":True,"grid.color":GRID,"axes.axisbelow":True,"axes.spines.top":False,
"axes.spines.right":False,"axes.titlesize":12,"axes.titleweight":"bold","legend.frameon":False})
sns.set_style("whitegrid")
BASE="https://raw.githubusercontent.com/johnfisher-ai/Statistics-Data-Science-AI-Visual-Book/main/data/"
In [2]:
import warnings; warnings.filterwarnings('ignore')
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
from sklearn.neighbors import KNeighborsClassifier, KNeighborsRegressor
from sklearn.tree import DecisionTreeClassifier
from sklearn.svm import SVC
from sklearn.linear_model import LogisticRegression, LinearRegression
from sklearn.metrics import accuracy_score
try: df = pd.read_excel('../../data/core-classification-and-regression-algorithms--patients.xlsx', sheet_name='Data')
except FileNotFoundError: df = pd.read_excel(BASE + 'core-classification-and-regression-algorithms--patients.xlsx', sheet_name='Data')
feat = ['age','resting_bp','cholesterol','max_hr','glucose','bmi']
X, y = df[feat], df['heart_disease']
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
print('loaded', df.shape)
loaded (700, 8)
CHALLENGE 1
Train a KNN classifier
Scale the features, fit KNN (k=15), and report test accuracy.
In [3]:
knn = make_pipeline(StandardScaler(), KNeighborsClassifier(15)).fit(X_tr, y_tr)
print(f'test accuracy = {accuracy_score(y_te, knn.predict(X_te)):.3f}')
test accuracy = 0.766
Solution. KNN measures distance, so the pipeline standardizes first; accuracy is scored on the held-out patients.
CHALLENGE 2
Read a tree's importances
Fit a shallow decision tree and print its most important feature.
In [4]:
tree = DecisionTreeClassifier(max_depth=4, random_state=0).fit(X_tr, y_tr)
imp = pd.Series(tree.feature_importances_, index=feat).sort_values(ascending=False)
print(imp.round(3).to_string()); print('top feature:', imp.idxmax())
age 0.415 bmi 0.199 max_hr 0.139 glucose 0.120 resting_bp 0.071 cholesterol 0.056 top feature: age
Solution. Age carries the strongest splits, which matches medical intuition for heart-disease risk.
CHALLENGE 3
Head-to-head
Compare logistic regression, a tree, and KNN by 5-fold cross-validated accuracy.
In [5]:
models = {'Logistic': make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000)),
'Tree': DecisionTreeClassifier(max_depth=4, random_state=0),
'KNN': make_pipeline(StandardScaler(), KNeighborsClassifier(15))}
for name,m in models.items(): print(f'{name:10s} {cross_val_score(m, X, y, cv=5).mean():.3f}')
Logistic 0.773 Tree 0.683 KNN 0.750
Solution. No model wins everywhere; cross-validate a few and let the numbers plus your constraints decide.
CHALLENGE 4
SVM: the kernel matters
Compare a linear SVM and an RBF SVM by cross-validated accuracy.
In [6]:
for k in ['linear','rbf']:
acc = cross_val_score(make_pipeline(StandardScaler(), SVC(kernel=k)), X, y, cv=5).mean()
print(f'SVM ({k:6s}) accuracy = {acc:.3f}')
SVM (linear) accuracy = 0.761 SVM (rbf ) accuracy = 0.766
Solution. The kernel sets the boundary shape: linear draws a straight divide, RBF bends to the data; try both.
CHALLENGE 5
The regression twin
Predict max heart rate (a number) with a KNN regressor and linear regression; compare R-squared.
In [7]:
Xr = df[['age','resting_bp','cholesterol','glucose','bmi']]; yr = df['max_hr']
for name,r in {'KNN reg': make_pipeline(StandardScaler(), KNeighborsRegressor(15)), 'Linear reg': LinearRegression()}.items():
print(f'{name:10s} R2 = {cross_val_score(r, Xr, yr, cv=5, scoring="r2").mean():.3f}')
KNN reg R2 = 0.274 Linear reg R2 = 0.356
Solution. KNN and the linear model each have a regression twin that uses the same idea (naive Bayes is the one method in this chapter without one); here they predict a number instead of a class.
Statistics, Data Science and AI: A Visual Handbook · © 2026 John Fisher