"Which points are most similar?" is the engine under a huge share of machine learning. But similarity is not one thing, straight-line closeness, shared features, and pointing in the same direction are different questions, and each has its own distance. Choosing the metric is a modeling decision, not a detail.
Standardize numeric features first, then match the metric to the data: Euclidean or Manhattan for continuous values, cosine for direction (text and embeddings), Jaccard or Hamming for binary flags, and Mahalanobis when features are correlated.
Euclidean, Manhattan & Minkowski
For continuous features, distance is geometry. Euclidean distance is the straight line between two points; Manhattan distance walks along the axes like a taxi on a grid. Both are special cases of the Minkowski distance with a parameter p.
In the notebook, scipy confirms it: from A = (1, 1) to B = (5, 4), Euclidean distance is
5.0 (a straight 3-4-5 triangle) and Manhattan is 7.0 (4 across plus 3 up).
Minkowski with p = 2 reproduces Euclidean and p = 1 reproduces Manhattan; larger p
weights the single biggest coordinate gap more heavily. Euclidean is the everyday default, while Manhattan is
steadier against outliers and often preferred in very high dimensions.
Cosine, Jaccard & Hamming
Not all data is a point in space. For direction, sets, and bit-strings, three non-Euclidean measures take over.
Cosine similarity is the cosine of the angle between two vectors, so it ignores magnitude entirely: in the notebook, u = (2, 1) and v = (6, 3) point the same way and score cosine 1.0, even though their Euclidean distance is a large 4.47. That is why cosine is the default for text, a long and a short document on the same topic should count as similar, and for the embedding vectors behind modern search. Jaccard similarity is shared features over total features present (2 of 4, or 0.5), ideal for sparse tags and market baskets; Hamming distance simply counts the positions that differ (2 of 5), natural for fixed-length codes.
Scaling First, and Mahalanobis
Two subtleties decide whether a distance is meaningful. First, scale: because Euclidean
distance squares each feature's difference, a feature measured in hundreds (price) drowns out one measured in single
digits (rating). The notebook shows the nearest neighbor changing after StandardScaler, so
always standardize numeric features first. Second, correlation: Euclidean
treats every direction alike, but real features are often correlated.
Mahalanobis distance divides by the covariance matrix, stretching distances along the data's thin directions. In the notebook, two points sit at nearly the same Euclidean radius, but the one lying across the correlated trend scores a Mahalanobis distance of 13.5 versus 3.0 for the one along it, the off-trend point is a genuine multivariate outlier, which Euclidean completely misses. That is why Mahalanobis is the standard ruler for anomaly detection.
Real-World Example: One Product, Three Neighbors
A catalog of 120 wireless headphones has numeric specs and binary feature flags. Ask for the product "most similar" to a given one and the answer changes with the ruler, a direct preview of how a recommender or a K-nearest-neighbors model behaves.
One row per product with numeric price, weight_g,
battery_hrs, rating, and binary flags noise_cancel, waterproof,
has_mic, fast_charge, foldable.
| Metric | Compares | Nearest product |
|---|---|---|
| Euclidean (standardized specs) | the size of numeric specs | H383 |
| Cosine (spec direction) | the balance of specs, not their size | H347 |
| Jaccard (feature flags) | shared yes/no features | H336 |
Three metrics, three different "most similar" products, all reasonable. Euclidean rewards products with similar numeric specs; cosine rewards a similar profile even at a different price point; Jaccard rewards the same feature set. There is no universally correct choice, the metric is where you encode what "similar" means for your problem, and standardizing the numeric columns first is what keeps the comparison fair.
Distance in Machine Learning & AI
Distance is not a side topic, it is the primitive under many of the most-used algorithms, all the way up to the vector search that powers today's AI assistants.
| Metric (this chapter) | Powers in ML / AI | Example |
|---|---|---|
| Euclidean / Manhattan | Nearest neighbors, K-means clustering | KNN classifier, customer segments |
| Cosine | Text and embedding similarity | semantic search, recommendation |
| Jaccard | Set / tag overlap, deduplication | market-basket, near-duplicate detection |
| Hamming | Codes and hashing | one-hot categories, locality-sensitive hashing |
| Mahalanobis | Multivariate anomaly detection | fraud and fault detection |
Modern AI runs on embeddings, dense vectors that place similar images, sentences, or products near each other. Everything downstream, semantic search, recommendation, and the retrieval in retrieval-augmented generation (RAG) that feeds a language model relevant context, is a nearest-neighbor query under a distance metric, almost always cosine on normalized vectors. At scale, exact search is too slow, so approximate nearest-neighbor indexes (HNSW, FAISS) trade a little accuracy for enormous speed. The rulers in this chapter are the literal mathematics of a vector database.
Compute every distance in Python
The companion notebook computes each metric with scipy.spatial.distance, draws the Manhattan
staircase, shows a nearest neighbor flipping after standardizing, visualizes cosine as an
angle and Hamming as mismatched bits, contrasts Euclidean and Mahalanobis on correlated data,
and finds one product's neighbor three different ways, walked through end to end.
View opens the rendered notebook instantly. Open in Colab runs it live. To run
locally, install numpy, pandas, scipy, scikit-learn,
matplotlib, and openpyxl.
🎓 Key Takeaways
- ✓Euclidean (L2) is straight-line distance, Manhattan (L1) walks the grid; both are Minkowski with p = 2 and p = 1.
- ✓Standardize numeric features first, otherwise the largest-scale feature dominates every distance.
- ✓Cosine measures direction, not magnitude, the default for text and embedding similarity.
- ✓Jaccard (shared / union) and Hamming (count of differences) handle binary and set data.
- ✓Mahalanobis corrects for feature covariance, catching multivariate outliers Euclidean misses.
Practice Challenges
Five short challenges. Try them with SciPy before checking the solutions.
Euclidean vs Manhattan
For A = (0, 0) and B = (3, 4), compute both distances and explain the difference.
euclidean and cityblock from scipy.spatial.distance.The Minkowski family
Show that Minkowski p = 1 equals Manhattan and p = 2 equals Euclidean.
minkowski(A, B, p).Standardize before distance
Show the nearest neighbor changes after scaling when features have very different ranges.
StandardScaler.Cosine ignores magnitude
Show two vectors pointing the same way have cosine similarity 1 but a large Euclidean distance.
Real data: two rulers
For the first product, find its nearest neighbor by scaled-Euclidean and by Jaccard on the flags.
cdist(..., "euclidean") and cdist(..., "jaccard").A fully-worked solutions notebook walks through all five challenges, each verified in code. Try them yourself first, then compare.
Quiz: Test Yourself
Eight quick questions on distance metrics. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.
With a workflow and a way to measure closeness in hand, we can meet the algorithms. Core Classification & Regression Algorithms covers the workhorses, K-nearest neighbors, decision trees, naive Bayes, SVM, and logistic regression, and when to reach for each.