←Back to the bookChapter 117 · Take the quiz & aim for 100%

Chapter 117 · Solutions

Anomaly Detection · Solutions

Five challenges, each verified in code.

Chapter 117 · Practice Challenges, Solved

Five anomaly-detection exercises, worked in scikit-learn on the Chapter 117 machine-sensor data.

import numpy as np, pandas as pd, matplotlib.pyplot as plt
import seaborn as sns   # seaborn = high-level statistical plots (heatmaps, pairplots, count/bar plots)
from matplotlib.colors import ListedColormap
EM="#4338ca"; DEEP="#3730a3"; LIGHT="#c7d2fe"; INK="#1a2138"; GRID="#e6e9f2"; RED="#ef4444"; AMBER="#d97706"; GREEN="#059669"; BLUE="#2563eb"; PUR="#9333ea"; GREY="#94a3b8"; SLATE="#475569"; ORG="#4338ca"; CYAN="#0891b2"
plt.rcParams.update({"figure.facecolor":"white","axes.facecolor":"white","figure.dpi":110,"font.size":11,
   "axes.edgecolor":GRID,"axes.grid":True,"grid.color":GRID,"axes.axisbelow":True,"axes.spines.top":False,
   "axes.spines.right":False,"axes.titlesize":12,"axes.titleweight":"bold","legend.frameon":False})
sns.set_style("whitegrid")
BASE="https://raw.githubusercontent.com/johnfisher-ai/Statistics-Data-Science-AI-Visual-Book/main/data/"

from sklearn.preprocessing import StandardScaler
from sklearn.covariance import EllipticEnvelope
from sklearn.ensemble import IsolationForest
from sklearn.neighbors import LocalOutlierFactor
from sklearn.metrics import recall_score, precision_score, accuracy_score
try: df = pd.read_excel('../../data/anomaly-detection--sensors.xlsx', sheet_name='Data')
except FileNotFoundError: df = pd.read_excel(BASE + 'anomaly-detection--sensors.xlsx', sheet_name='Data')
feat = ['temperature','vibration','pressure','rotation_rpm']
X = df[feat]; y = df['label']; Xs = StandardScaler().fit_transform(X)
rate = float(y.mean()); print('anomaly rate', round(rate,3))

anomaly rate 0.051

CHALLENGE 1

z-score rule

Flag readings over 3 SDs from the mean and report recall.

flag = ((np.abs((X - X.mean())/X.std()) > 3).any(axis=1)).astype(int)
print('recall', round(recall_score(y, flag),2), '| precision', round(precision_score(y, flag),2))

recall 0.6 | precision 1.0

Solution. The 3-sigma rule catches every obvious anomaly with perfect precision but recall only 0.60, it is blind to multivariate anomalies that look normal on each single feature.

CHALLENGE 2

Mahalanobis

Use EllipticEnvelope and report recall; note what it adds over the z-score.

ee = EllipticEnvelope(contamination=rate, random_state=0).fit(Xs)
flag = (ee.predict(Xs) == -1).astype(int)
print('recall', round(recall_score(y, flag),2))

recall 1.0

Solution. Recall jumps to 1.00: by modeling the covariance, Mahalanobis distance flags points that break the temperature-vibration correlation, the subtle anomalies the z-score missed.

CHALLENGE 3

Isolation Forest

Fit an Isolation Forest, extract anomaly scores, and report recall.

iso = IsolationForest(contamination=rate, random_state=0).fit(Xs)
flag = (iso.predict(Xs) == -1).astype(int)
score = -iso.score_samples(Xs)
print('recall', round(recall_score(y, flag),2), '| mean score normal vs anomaly:', round(score[y==0].mean(),3), round(score[y==1].mean(),3))

recall 0.63 | mean score normal vs anomaly: 0.435 0.577

Solution. Isolation Forest isolates globally-odd points in a few random splits, assumption-free and scalable; anomalies get clearly higher scores. It catches the obvious anomalies (the correlation-breakers need a density or covariance method).

CHALLENGE 4

Local Outlier Factor

Use LOF and note which kind of anomaly it specializes in.

lof = LocalOutlierFactor(n_neighbors=20, contamination=rate)
flag = (lof.fit_predict(Xs) == -1).astype(int)
sub = (y==1) & (df['vibration']<1.6)
print('recall', round(recall_score(y, flag),2), '| subtle caught', int(((flag==1)&sub).sum()), 'of', int(sub.sum()))

recall 0.47 | subtle caught 12 of 12

Solution. LOF compares each point's local density to its neighbors', so it excels at the subtle correlation-breakers (all 12 caught) even though it can miss a tight cluster of obvious anomalies. It is the opposite specialist to Isolation Forest.

CHALLENGE 5

Why not accuracy?

Show why accuracy is the wrong metric, then combine two detectors for high recall.

print('accuracy of a do-nothing detector (flag none):', round(accuracy_score(y, np.zeros_like(y)),3))
ee = (EllipticEnvelope(contamination=rate, random_state=0).fit_predict(Xs)==-1).astype(int)
lof = (LocalOutlierFactor(n_neighbors=20, contamination=rate).fit_predict(Xs)==-1).astype(int)
combo = ((ee==1) | (lof==1)).astype(int)
print('combined (elliptic OR LOF) recall:', round(recall_score(y, combo),2), '| precision:', round(precision_score(y, combo),2))

accuracy of a do-nothing detector (flag none):

 0.949
combined (elliptic OR LOF) recall: 1.0 | precision: 0.65

Solution. Because anomalies are only 5% of the data, a detector that flags nothing scores 95% accuracy while catching zero, so accuracy is useless here; use recall and precision. Combining complementary detectors (flag if either fires) raises recall by covering each other's blind spots.

---

Statistics, Data Science and AI: A Visual Handbook · © 2026 John Fisher

←Back to the bookChapter 117 · Take the quiz & aim for 100%