←Back to the bookChapter 115 · Dimensionality Reduction

Chapter 115 · Dimensionality Reduction

Dimensionality Reduction

Compressing eight correlated nutrition columns into a few informative axes with PCA, reading the scree plot, loadings, and biplot, using the components to compress and denoise, and contrasting the linear PCA with the nonlinear t-SNE and UMAP, all library-first with scikit-learn, and heavily visual.

Statistics, Data Science and AI: A Visual Handbook · John Fisher · 2026

Dimensionality Reduction: many columns into a few

Real datasets are wide: dozens of correlated columns that are hard to plot, slow to model, and noisy. Dimensionality reduction compresses them into a handful of informative axes while keeping most of the information. This notebook uses PCA (principal component analysis) on packaged-food nutrition, reads its scree plot, loadings, and biplot, uses it to compress and denoise, and contrasts it with the nonlinear t-SNE. Library-first with scikit-learn.

import numpy as np, pandas as pd, matplotlib.pyplot as plt
import seaborn as sns   # seaborn = high-level statistical plots (heatmaps, pairplots, count/bar plots)
from matplotlib.colors import ListedColormap
EM="#4338ca"; DEEP="#3730a3"; LIGHT="#c7d2fe"; INK="#1a2138"; GRID="#e6e9f2"; RED="#ef4444"; AMBER="#d97706"; GREEN="#059669"; BLUE="#2563eb"; PUR="#9333ea"; GREY="#94a3b8"; SLATE="#475569"; ORG="#4338ca"; CYAN="#0891b2"
plt.rcParams.update({"figure.facecolor":"white","axes.facecolor":"white","figure.dpi":110,"font.size":11,
   "axes.edgecolor":GRID,"axes.grid":True,"grid.color":GRID,"axes.axisbelow":True,"axes.spines.top":False,
   "axes.spines.right":False,"axes.titlesize":12,"axes.titleweight":"bold","legend.frameon":False})
sns.set_style("whitegrid")
BASE="https://raw.githubusercontent.com/johnfisher-ai/Statistics-Data-Science-AI-Visual-Book/main/data/"

from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.manifold import TSNE
pd.set_option('display.max_columns', 30)

DEMO 1 · TOO MANY COLUMNS

Eight correlated nutrition features

Load foods and see how tangled the columns are.

try: df = pd.read_excel('../../data/dimensionality-reduction--foods.xlsx', sheet_name='Data')
except FileNotFoundError: df = pd.read_excel(BASE + 'dimensionality-reduction--foods.xlsx', sheet_name='Data')
num = ['calories','fat_g','satfat_g','protein_g','carbs_g','sugar_g','fiber_g','sodium_mg']
Xs = StandardScaler().fit_transform(df[num])   # PCA needs standardized inputs
fig, ax = plt.subplots(figsize=(7,5.6))
sns.heatmap(df[num].corr(), annot=True, fmt='.2f', cmap='coolwarm', center=0, ax=ax)
ax.set_title('Correlation among the 8 nutrition columns'); plt.tight_layout(); plt.show()

Why reduce? These eight columns are heavily correlated, calories move with fat and carbs, sugar moves against fiber, and so on. That redundancy means the data does not really live in 8 dimensions; a few underlying factors (energy density, sweet vs savory) drive everything. Dimensionality reduction finds those factors. As always with distance- and variance-based methods, we standardize first so a large-range column like sodium does not dominate purely because of its units.

DEMO 2 · PCA AND THE SCREE PLOT

How many dimensions do we really need?

PCA ranks new axes by how much variance each explains.

pca = PCA().fit(Xs)
ev = pca.explained_variance_ratio_
cum = np.cumsum(ev)
fig, ax = plt.subplots(figsize=(8,4.4))
ax.bar(range(1,9), ev, color=EM, alpha=0.85, label='each PC')
ax.plot(range(1,9), cum, 'o-', color=RED, lw=2.2, label='cumulative')
ax.axhline(0.9, color=GREY, ls='--'); ax.set(xlabel='principal component', ylabel='share of variance', title='Scree plot: variance explained per component'); ax.legend()
plt.tight_layout(); plt.show()
print('variance explained: PC1 %.0f%%  PC2 %.0f%%  PC3 %.0f%%' % (ev[0]*100, ev[1]*100, ev[2]*100))
print('first 2 PCs: %.0f%% of all variance | first 3: %.0f%%' % (cum[1]*100, cum[2]*100))

variance explained: PC1 42%  PC2 34%  PC3 19%
first 2 PCs: 76% of all variance | first 3: 95%

Reading the scree plot. PCA rebuilds the data on new perpendicular axes (principal components) ordered so the first captures the most variance, the second the next most, and so on. Here PC1 explains 42% and PC2 34%, so a 2D picture keeps 76% of everything; adding PC3 reaches 95%. The 8-column dataset was really only about 3 dimensions of information. The 'elbow' in the scree plot, and the point where the cumulative curve crosses 90%, tell you how many components to keep.

DEMO 3 · THE 2D MAP

Eight dimensions on one page

Project every food onto PC1 and PC2 and plot it.

P = PCA(n_components=2).fit_transform(Xs)
fig, ax = plt.subplots(figsize=(7.6,5.6))
for t,c in zip(['Energy-dense','Sweet','Savory'], [EM, PUR, GREEN]):
    m = df['food_type']==t
    ax.scatter(P[m,0], P[m,1], s=40, c=c, edgecolor='white', label=t)
ax.set(xlabel='PC1 (42% of variance)', ylabel='PC2 (34% of variance)', title='Foods projected onto their first two principal components'); ax.legend()
plt.tight_layout(); plt.show()

A readable map. Each food is now a single dot in a 2D plane, its coordinates are its scores on PC1 and PC2. Even though PCA never saw the food_type label (it is unsupervised), coloring the dots by type shows the categories fall in distinct regions: the compression preserved the real structure. This is the everyday magic of PCA, turning an un-plottable 8-column table into one honest picture you can actually look at.

DEMO 4 · LOADINGS AND THE BIPLOT

What do the new axes mean?

Loadings show which original columns build each PC.

pca2 = PCA(n_components=2).fit(Xs)
comp = pca2.components_
load = pd.DataFrame(comp.T, index=num, columns=['PC1','PC2']).round(2)
print(load.to_string())
fig, ax = plt.subplots(figsize=(7.6,5.8))
ax.scatter(P[:,0], P[:,1], s=18, c=GREY, alpha=0.4)
for i,name in enumerate(num):
    ax.arrow(0,0, comp[0,i]*3.2, comp[1,i]*3.2, color=EM, width=0.006, head_width=0.12, length_includes_head=True)
    ax.text(comp[0,i]*3.6, comp[1,i]*3.6, name, color=DEEP, fontsize=9, ha='center', fontweight='bold')
ax.axhline(0,color=GRID); ax.axvline(0,color=GRID); ax.set(xlabel='PC1', ylabel='PC2', title='Biplot: how each nutrient loads onto PC1 and PC2')
plt.tight_layout(); plt.show()

            PC1   PC2
calories   0.43  0.35
fat_g      0.35  0.44
satfat_g   0.35  0.43
protein_g -0.27  0.37
carbs_g    0.47 -0.13
sugar_g    0.35 -0.33
fiber_g   -0.38  0.39
sodium_mg -0.11  0.29

Interpreting the axes. Loadings are the recipe for each component. PC1 loads high on calories, carbs, and sugar and low on fiber and protein, it is a 'carb-and-calorie load' axis. PC2 loads high on fat, saturated fat, and protein, it is a 'richness/fat' axis. In the biplot, each arrow is a nutrient: arrows pointing the same way are correlated, opposite arrows are anti-correlated (sugar vs fiber), and a food sitting far along an arrow is high in that nutrient. This is what makes PCA interpretable, the axes are readable combinations of the originals, not a black box.

DEMO 5 · COMPRESSION AND DENOISING

Keep a few PCs, throw the rest away

Reconstruct the data from only the top components.

errs = []
for k in range(1,9):
    pk = PCA(n_components=k).fit(Xs)
    recon = pk.inverse_transform(pk.transform(Xs))
    errs.append(np.mean((Xs - recon)**2))
fig, ax = plt.subplots(figsize=(7.6,4.2))
ax.plot(range(1,9), errs, 'o-', color=EM, lw=2.4)
ax.set(xlabel='components kept', ylabel='reconstruction error (MSE)', title='Fewer components = lossy compression')
ax.axvline(3, color=RED, ls='--'); plt.tight_layout(); plt.show()
print('keeping 3 of 8 components reconstructs the data with error %.3f and retains 95%% of variance' % errs[2])

keeping 3 of 8 components reconstructs the data with error 0.053 and retains 95% of variance

A dial between size and fidelity. Reconstructing each food from only its top-k component scores is lossy compression: fewer components mean a smaller representation but a larger reconstruction error. The curve drops steeply then flattens, keeping just 3 of the 8 components rebuilds the table almost perfectly (95% of the variance). Because the discarded components are mostly noise, this step also denoises the data, which is exactly why PCA is a standard preprocessing step before clustering or supervised models on wide, correlated data.

DEMO 6 · NONLINEAR: t-SNE

When straight axes are not enough

t-SNE unfolds curved structure for visualization.

emb = TSNE(n_components=2, perplexity=30, random_state=0, init='pca').fit_transform(Xs)
fig, ax = plt.subplots(1, 2, figsize=(12,5))
for t,c in zip(['Energy-dense','Sweet','Savory'], [EM, PUR, GREEN]):
    m = df['food_type']==t
    ax[0].scatter(P[m,0], P[m,1], s=28, c=c, edgecolor='white', label=t)
    ax[1].scatter(emb[m,0], emb[m,1], s=28, c=c, edgecolor='white', label=t)
ax[0].set_title('PCA (linear)'); ax[1].set_title('t-SNE (nonlinear)'); ax[0].legend()
plt.tight_layout(); plt.show()

Linear vs nonlinear. PCA can only rotate and stretch, it finds straight axes, which makes it fast, reversible, and interpretable. t-SNE (and its faster cousin UMAP) instead preserve local neighborhoods, unfolding curved manifolds into tidy islands that often separate groups more crisply. The trade-offs: t-SNE and UMAP are for visualization only (the axes have no units, distances between clusters are not meaningful, and you cannot project new points reversibly). Rule of thumb: reach for PCA to compress, denoise, and interpret; reach for t-SNE/UMAP to make a striking 2D picture of high-dimensional data.

Dimensionality reduction, in one view

• Wide, correlated data does not need all its columns, a few latent factors drive it.
• PCA builds ranked perpendicular axes; the scree plot and the 90% cumulative line tell you how many to keep.
• Loadings / biplot make the components interpretable (which original columns build each axis).
• Keeping the top few components compresses and denoises, a standard preprocessing step.
• PCA is linear and interpretable; t-SNE / UMAP are nonlinear and for visualization only.

---

Statistics, Data Science and AI: A Visual Handbook · © 2026 John Fisher

←Back to the bookChapter 115 · Dimensionality Reduction