For most of this book a human wrote the rule: compute a mean, fit a line, run a test. Machine learning flips that around, you show the computer examples and it learns the rule itself, then applies it to data it has never seen. That single shift powers recommendation feeds, fraud alerts, medical triage, and the large language models behind modern AI.
If your data has a label to predict, it is supervised (classification for a category, regression for a number); if it has no label, it is unsupervised; if an agent learns from reward, it is reinforcement learning.
The Three Paradigms of Learning
Almost every machine-learning method belongs to one of three families, distinguished by what kind of feedback the algorithm gets. This one picture is the map for the rest of the section.
The vast majority of applied work is supervised, because so many valuable questions come with labeled history: past emails marked spam or not, past loans that defaulted or not, past customers who churned or not. Unsupervised learning is the tool for exploration when no label exists, and reinforcement learning shines when an agent must act in a changing environment. The notebook runs one small example of each on the same data.
Supervised Learning: Classification vs Regression
Supervised learning always has the same shape: a table of features (the inputs, written X) and a label (the target, written y). The model learns the mapping from X to y on training examples, then predicts y for new rows. It splits in two by the type of the label.
In the notebook, a LogisticRegression classifier predicts the category
churned (yes/no) and reaches 77% accuracy on customers it never trained on, against 71% for a rule that always
predicts "stayed"; a
LinearRegression regressor predicts the number monthly_spend and scores
R² = 0.83 on the test set. Same three-step recipe every time, split, fit,
evaluate on held-out data, changing only the kind of answer. And note the golden rule already in play: we
always judge a model on data it has not seen, because memorizing the training set is not learning.
Statistics vs Machine Learning
You have already done machine learning, a fitted regression is a supervised model. What changes in ML is the emphasis: from explaining a process to predicting new outcomes.
| Aspect | Classical statistics | Machine learning |
|---|---|---|
| Main goal | inference: explain, with uncertainty | prediction: generalize to new data |
| Key question | which effects are real, and how big? | how accurate is it on unseen cases? |
| Prized output | coefficients, p-values, intervals | held-out accuracy, error, loss |
| Validation | model assumptions, diagnostics | train / test split, cross-validation |
| Typical tool | statsmodels | scikit-learn |
The notebook makes this vivid on the churn data: the statistics view fits a logistic model and reads its coefficients, and short tenure, lower income and frequent support calls are significantly associated with higher churn (tiny p-values, from observational data); the machine-learning view cross-validates the same model and reports a single number, 79% accuracy on unseen customers, where always guessing "stayed" would score about 71%. Neither is better, they answer different questions, and a strong practitioner moves fluently between "why?" and "how well will it predict?"
Real-World Example: Supervised and Unsupervised on One Table
A telecom has 600 customers with six numeric features and a churn label. The same table answers a supervised question (who will churn?) and an unsupervised one (what natural segments exist?), the two most common tasks in commercial ML.
One row per customer with age, income_k,
tenure_months, num_products, monthly_spend, and support_calls,
plus the label churned (1 = left last quarter, 0 = stayed).
Run unsupervised on the numeric features and KMeans finds three segments with no
label at all. They are broad groups whose edges overlap, as the 40 customers drawn here from the 600 show:
| Discovered segment | Size | Avg income / spend | Churn rate |
|---|---|---|---|
| Budget | 269 | ~40k / $43.5, 1.5 products, 15 mo | 50% |
| Standard | 242 | ~73k / $61.0, 2.4 products, 39 mo | 15% |
| Premium | 89 | ~101k / $89.6, 4.5 products, 50 mo | 3% |
The two tasks reinforce each other. Unsupervised clustering found the segments blind, and they still line up closely with the supervised label. The budget segment (short tenure, few products) churns at 50% and holds 135 of the 174 churners, about three in four, while the premium segment barely leaves. That is the practical payoff of the taxonomy. One dataset seen through two lenses points retention work at the budget segment, where most of the churn is.
The Paradigms in Modern AI
The three families are not just textbook categories, they are the scaffolding of every frontier AI system. The same words, scaled up with deep neural networks and enormous datasets, describe how today's models are built.
| Paradigm (this chapter) | At the frontier it becomes | Example |
|---|---|---|
| Supervised | Labeled deep learning | image classifiers, medical diagnosis from scans |
| Unsupervised / self-supervised | Pretraining on raw data | an LLM learning to predict the next token on the web |
| Reinforcement | Learning from feedback | game agents; RLHF aligning chatbots to human preference |
| Regression / classification | The two output heads | a network ending in a number or a class probability |
A modern large language model is a tour of all three paradigms at once. It is pretrained by self-supervised learning, predicting the next word on trillions of tokens with no human labels, which is unsupervised learning in disguise. It is then fine-tuned with supervised examples of good answers, and finally aligned with reinforcement learning from human feedback (RLHF), an agent optimizing a reward built from human preferences. The vocabulary of this one chapter, features and labels, classification and regression, clustering, reward, is exactly the language you need to read a frontier AI paper.
Run all three paradigms in Python
The companion notebook trains a churn classifier and a spend regressor with
scikit-learn, finds customer segments with K-means, teaches a
reinforcement-learning agent to pick the best offer from reward, and contrasts the
statsmodels inference view with the cross-validated ML view, every cell explained.
View opens the rendered notebook instantly (no setup). Open in Colab runs &
edits it live in your browser. To run locally, install numpy, pandas,
scikit-learn, statsmodels, matplotlib, seaborn, and
openpyxl.
🎓 Key Takeaways
- ✓Machine learning learns rules from data instead of being programmed with them, then applies them to unseen cases.
- ✓Three paradigms: supervised (labeled examples), unsupervised (structure, no label), reinforcement (a policy from reward).
- ✓Supervised splits by label type: classification predicts a category, regression predicts a number, same split-fit-evaluate recipe.
- ✓Always evaluate on held-out data: a model that only fits the training set has memorized, not learned.
- ✓Statistics vs ML is a difference of emphasis: inference (explain) versus prediction (generalize); frontier AI uses all three paradigms at once.
Practice Challenges
Five short challenges on the customer table. Try them with scikit-learn before checking the solutions.
Features vs label
Separate the input features from the target and print the shape of each.
X = df[feat], y = df["churned"].Train / test split and accuracy
Hold out 30%, train a logistic classifier, and report accuracy on the unseen part.
train_test_split then accuracy_score.Regression instead
Predict monthly_spend (a number) and report the test R².
LinearRegression().score(X_test, y_test).Cluster into segments
Run K-means with k = 3 on the numeric features; print the mean income and spend per cluster.
KMeans(n_clusters=3).fit_predict(...) then groupby.Explain vs predict
Fit a statsmodels logit to find the strongest churn predictor, then cross-validate for a prediction score.
cross_val_score(..., cv=5).A fully-worked solutions notebook walks through all five challenges, each verified in code. Try them yourself first, then compare.
Quiz: Test Yourself
Eight quick questions on the machine-learning taxonomy. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.
You now know what the field contains. The Machine Learning Workflow turns the taxonomy into a disciplined process, train/validation/test splits, cross-validation, hyperparameter tuning, pipelines, and the leakage traps that quietly inflate every beginner's accuracy.