Chapter 117 · Solutions
Model Interpretability & Explainability · Solutions
Five challenges, each verified in code.
Chapter 117 · Practice Challenges, Solved¶
Five exercises on interpretability, worked in scikit-learn, shap, and lime on the Chapter 117 credit table.
In [1]:
import numpy as np, pandas as pd, matplotlib.pyplot as plt
import seaborn as sns # seaborn = high-level statistical plots (heatmaps, pairplots, count/bar plots)
from matplotlib.colors import ListedColormap
EM="#4338ca"; DEEP="#3730a3"; LIGHT="#c7d2fe"; INK="#1a2138"; GRID="#e6e9f2"; RED="#ef4444"; AMBER="#d97706"; GREEN="#059669"; BLUE="#2563eb"; PUR="#9333ea"; GREY="#94a3b8"; SLATE="#475569"; ORG="#4338ca"; CYAN="#0891b2"
plt.rcParams.update({"figure.facecolor":"white","axes.facecolor":"white","figure.dpi":110,"font.size":11,
"axes.edgecolor":GRID,"axes.grid":True,"grid.color":GRID,"axes.axisbelow":True,"axes.spines.top":False,
"axes.spines.right":False,"axes.titlesize":12,"axes.titleweight":"bold","legend.frameon":False})
sns.set_style("whitegrid")
BASE="https://raw.githubusercontent.com/johnfisher-ai/Statistics-Data-Science-AI-Visual-Book/main/data/"
In [2]:
import warnings; warnings.filterwarnings('ignore')
from sklearn.model_selection import train_test_split
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.inspection import permutation_importance, partial_dependence
import shap, lime, lime.lime_tabular
try: df = pd.read_excel('../../data/model-interpretability-and-explainability--credit.xlsx', sheet_name='Data')
except FileNotFoundError: df = pd.read_excel(BASE + 'model-interpretability-and-explainability--credit.xlsx', sheet_name='Data')
feat = ['credit_score','debt_to_income','income_k','loan_amount_k','num_late_payments','employment_years']
X, y = df[feat], df['default']
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
model = GradientBoostingClassifier(random_state=0).fit(X_tr, y_tr)
print('model test accuracy', round(model.score(X_te,y_te),3))
model test accuracy 0.71
CHALLENGE 1
Permutation importance
Rank the features by permutation importance and name the most important.
In [3]:
r = permutation_importance(model, X_te, y_te, n_repeats=20, random_state=0)
imp = pd.Series(r.importances_mean, index=feat).sort_values(ascending=False)
print(imp.round(4).to_string()); print('top:', imp.idxmax())
credit_score 0.0745 income_k 0.0242 debt_to_income 0.0217 num_late_payments 0.0095 loan_amount_k 0.0092 employment_years -0.0118 top: credit_score
Solution. Shuffling credit score hurts accuracy most, about three times as much as shuffling income or debt-to-income, so the model leans on it most.
CHALLENGE 2
Partial dependence direction
Show the model's predicted risk falls as credit score rises.
In [4]:
pd_res = partial_dependence(model, X_te.astype(float), ['credit_score'])
vals = pd_res['average'][0]; grid = pd_res['grid_values'][0]
print(f'partial dependence at low score ({grid[0]:.0f}) = {vals[0]:.3f}')
print(f'partial dependence at high score ({grid[-1]:.0f}) = {vals[-1]:.3f}')
print('falls as credit score rises:', vals[-1] < vals[0])
partial dependence at low score (581) = 1.326 partial dependence at high score (795) = -1.566 falls as credit score rises: True
Solution. Partial dependence decreases from low to high credit score, so the model learned the expected direction. The values are on the log-odds scale, which is why they run below zero.
CHALLENGE 3
Global SHAP importance
Rank features by mean absolute SHAP value.
In [5]:
sv = shap.TreeExplainer(model)(X_te)
shap_imp = pd.Series(np.abs(sv.values).mean(axis=0), index=feat).sort_values(ascending=False)
print(shap_imp.round(3).to_string())
credit_score 0.694 debt_to_income 0.436 loan_amount_k 0.356 num_late_payments 0.204 income_k 0.170 employment_years 0.107
Solution. Mean |SHAP| gives a global importance ranking. It agrees with permutation importance that credit score comes first, and disagrees lower down, where income is second by permutation importance and fifth by mean |SHAP|.
CHALLENGE 4
SHAP for one applicant
For the highest-risk applicant, find the feature contributing most to the default prediction.
In [6]:
i = int(np.argmax(model.predict_proba(X_te)[:,1]))
contrib = pd.Series(sv.values[i], index=feat).sort_values(ascending=False)
print(f'applicant predicted default prob = {model.predict_proba(X_te.iloc[[i]])[0,1]:.2f}')
print('top risk-increasing feature:', contrib.idxmax(), f'(SHAP {contrib.max():+.2f})')
applicant predicted default prob = 0.91 top risk-increasing feature: loan_amount_k (SHAP +1.56)
Solution. The largest positive SHAP value is the feature pushing this applicant most toward default.
CHALLENGE 5
LIME agreement
Explain the same applicant with LIME and compare its top driver to SHAP.
In [7]:
lt = lime.lime_tabular.LimeTabularExplainer(X_tr.values, feature_names=feat, class_names=['repaid','default'], mode='classification', random_state=0)
exp = lt.explain_instance(X_te.values[i], model.predict_proba, num_features=6)
top = max(exp.as_list(), key=lambda t: t[1])
print('LIME top risk driver:', top)
LIME top risk driver: ('credit_score <= 634.75', 0.2618473601997385)
Solution. LIME (a local linear surrogate) puts credit score first, while SHAP's largest push for this applicant is the loan amount. The two share the same top three features in a different order, a consistency check on the explanation that does not prove either one right.
Statistics, Data Science and AI: A Visual Handbook · © 2026 John Fisher