Some of the most valuable data points are the ones that do not fit: the fraudulent charge, the failing machine, the intruder on the network. Anomaly detection finds these rare outliers, and it is usually unsupervised, because you rarely have labeled examples of every way things can go wrong. You must recognize "abnormal" from a model of "normal" alone.
Model "normal," then flag the rare deviations: z-score (one feature), Mahalanobis (accounts for correlation), Isolation Forest (scalable, assumption-free), and LOF (local density), each catches a different kind of anomaly, so match the method to the anomaly you fear, or combine them.
What Is an Anomaly?
An anomaly is a point that departs from the pattern of the rest. Two things make detection hard. Anomalies are rare, often 1 to 5% of the data, so plain accuracy is worthless: a detector that flags nothing scores 95% accuracy on our sensors while catching zero faults. That is why we score with recall (of the true anomalies, how many did we catch?) and precision. And crucially, anomalies come in two flavors.
Obvious anomalies are extreme on some feature (top-right: both temperature and vibration high), any method catches them. Subtle multivariate anomalies look normal on each feature but break the relationship between features (high temperature yet low vibration, off the normal diagonal). Catching those is the real test, and it decides which method you need.
Statistical Methods: z-score & Mahalanobis
The oldest approach is statistical: model normal, then flag the improbable. The z-score rule flags any value more than three standard deviations from its column mean, the familiar 3-sigma rule. It is simple and catches every obvious anomaly, but it is half-blind: judging each feature alone, it caught 0 of the 12 subtle anomalies. A univariate rule simply cannot see a problem that lives in the relationship between two features.
The fix is to go multivariate. Mahalanobis distance (via EllipticEnvelope) fits one Gaussian to the data and measures each point's distance in units of the data's own covariance. Because it knows temperature and vibration should move together, a point off that diagonal is far in Mahalanobis terms even when it is ordinary on each axis, so it caught every anomaly, obvious and subtle (recall 1.00). The price: it assumes the data is roughly Gaussian, a strong assumption real data often violates.
ML Detectors: Isolation Forest & LOF
Two machine-learning detectors drop the Gaussian assumption and attack the problem geometrically, from opposite directions.
Isolation Forest builds many random trees, each splitting on a random feature at a random value. Anomalies, being few and different, get isolated in just a few splits, so a short average path length becomes a high anomaly score. It assumes nothing about the data's shape, scales to millions of rows and many features, and needs almost no tuning, which makes it the practical default for large, high-dimensional problems. Its weakness: axis-parallel cuts struggle with subtle correlation-breakers that are not globally extreme.
Local Outlier Factor (LOF) asks a relative question: is this point in a much sparser region than its neighbors are? That makes it superb at local anomalies, a point stranded off the correlation line sits in empty space, so LOF flagged all 12 subtle anomalies. Curiously it can miss a tight cluster of obvious anomalies (they look dense relative to each other). Isolation Forest and LOF are near-opposites, one hunts globally isolated points, the other locally odd ones, and that contrast is exactly the lesson.
Real-World Example: Machine Fault Detection
Predictive maintenance is a classic anomaly-detection job: watch a fleet of machines and flag the few whose sensors drift into trouble, before they fail.
590 machines with temperature, vibration,
pressure, and rotation_rpm. About 5% are faulty. A label column gives the
ground truth, used only to score the detectors, never to train them.
Running all four detectors on the same data lays bare the central truth of the field, no single method wins on every anomaly type:
| Detector | Recall | Its strength |
|---|---|---|
| z-score (univariate) | 0.60 | simple, perfect precision on extreme single-feature outliers |
| Mahalanobis (elliptic) | 1.00 | accounts for correlation, best when data is roughly Gaussian |
| Isolation Forest | 0.63 | assumption-free, scalable, the go-to default |
| LOF (density) | 0.47 | catches locally-odd points others miss |
The practical move: choose by the anomaly you most fear, or combine complementary detectors (flag if any fires, which pushed recall to 1.00 here) and tune the contamination threshold to trade false alarms against misses.
Setting the threshold when you have no labels. Every detector above was handed
contamination = y.mean(), the true anomaly rate, which you only know here because it is a teaching
set. In the real unlabeled case you set the cutoff two other ways. A budget: if the team
can investigate the worst 3% of units, score everything with score_samples and flag the lowest 3%
(in the notebook that is 18 of 590 units). Or a gap: sort the scores and cut at a visible
jump between the normal bulk and a straggling tail. Neither needs ground truth; when a few confirmed cases
eventually arrive, use them to sanity-check the flag rate, not to set it.
Anomaly Detection in Machine Learning & AI
Spotting the outlier is one of the most widely deployed unsupervised tasks in industry.
| Method / idea | Where it is used |
|---|---|
| Isolation Forest | fraud detection, predictive maintenance, high-dimensional monitoring |
| LOF / density | local outliers in spatial, sensor, and network data |
| Mahalanobis | quality control, multivariate process monitoring |
| Autoencoder reconstruction error | deep anomaly detection on images, logs, and time series |
| Recall / precision, thresholds | tuning the alarm rate in any rare-event system |
Anomaly detection guards the systems AI runs on. In production, drift and outlier detection watch a model's inputs and flag when live data stops resembling the training data, the early-warning signal that a model is about to fail silently. Deep learning brings its own twist: an autoencoder (Chapter 115's idea) is trained to reconstruct normal data, so a large reconstruction error marks an anomaly, the standard approach for images, logs, and time series. And the field's core discipline, that anomalies are rare, unlabeled, and diverse, so you must reason about recall and precision rather than accuracy, is exactly the mindset needed for fraud, cybersecurity, and AI-safety monitoring, where the rare event is the whole point.
Detect anomalies in Python
The companion notebook hunts machine faults with the z-score, Mahalanobis (EllipticEnvelope), Isolation Forest, and LOF, scores each with recall and precision, and shows head-to-head why no single detector wins, each cell explained.
View opens the rendered notebook instantly. Open in Colab runs it live. To run
locally, install numpy, pandas, scikit-learn, matplotlib, and
openpyxl.
🎓 Key Takeaways
- ✓Anomalies are rare and usually unlabeled, judge detectors by recall and precision, never accuracy.
- ✓z-score (univariate) is simple but blind to anomalies that live in the relationship between features.
- ✓Mahalanobis / EllipticEnvelope accounts for correlation, excellent when data is roughly Gaussian.
- ✓Isolation Forest assumes nothing and scales, the default; LOF catches locally-odd points.
- ✓No single detector wins, match the method to the anomaly you fear, or combine detectors and tune the threshold.
Practice Challenges
Five short challenges. Try them with scikit-learn before checking the solutions.
z-score rule
Flag readings over 3 SDs from the mean and report the recall.
(np.abs((X-X.mean())/X.std()) > 3).any(axis=1).Mahalanobis
Use EllipticEnvelope and report recall; note what it adds over the z-score.
EllipticEnvelope(contamination=rate).fit_predict(Xs).Isolation Forest
Fit an Isolation Forest, extract anomaly scores, and report recall.
IsolationForest(contamination=rate); -score_samples.Local Outlier Factor
Use LOF and note which kind of anomaly it specializes in.
LocalOutlierFactor(n_neighbors=20, contamination=rate).fit_predict(Xs).Why not accuracy?
Show why accuracy is the wrong metric, then combine two detectors for high recall.
A fully-worked solutions notebook walks through all five challenges, each verified in code. Try them yourself first, then compare.
Quiz: Test Yourself
Eight quick questions on anomaly detection. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.
That completes unsupervised learning, clustering, dimensionality reduction, association rules, and anomaly detection, all learning structure from unlabeled data. Next we leave both labels and fixed datasets behind. Reinforcement Learning Primer opens Reinforcement Learning, where an agent learns by acting in an environment and collecting rewards.