Two models can share an R² of 0.70. One is trustworthy; the other systematically under-predicts half your cases and has confidence intervals that are quietly wrong. The only way to tell them apart is to look at the residuals, what the model got wrong. Diagnostics are that look.
Steps 1 and 4, check the data, then re-check the residuals, live here. A model that fails them is not ready to interpret, no matter how high its R². Diagnose, remedy, re-fit.
The LINE Assumptions and Their Diagnostics
Each assumption has a plot that reveals it and a formal test that confirms it. Learn the shapes first, one residual-versus-fitted plot and one Q-Q plot catch most problems at a glance.
A healthy fit shows a shapeless band; every other panel is a specific alarm. Here is the full reference:
| Assumption | Diagnostic plot | Formal test | If violated |
|---|---|---|---|
| Linearity | residuals vs fitted (want no curve) | rainbow / RESET test | add a squared term, transform, or a curved model |
| Independence | residuals vs order / time | Durbin-Watson (≈ 2 is good) | time-series or panel methods (see Econometrics & Panel Data) |
| Normality | Q-Q plot of residuals (want a straight line) | Shapiro-Wilk, Jarque-Bera | transform y; large n is fairly forgiving |
| Equal variance | scale-location / residuals vs fitted (want no funnel) | Breusch-Pagan, White | log y, weighted least squares, robust SEs |
One habit worth building: always plot before you test. A test gives a yes or no; the plot tells you how the assumption fails, which is what points you to the right fix.
Outliers, Leverage, and Influence
Not every unusual point is a problem. Three ideas keep them straight. An outlier has a surprising y (a large residual). A high-leverage point has an extreme x, far from the others. Only when a point is both does it have influence, the power to move the fitted line.
The amber point is a clear outlier but sits in the middle of the x-range, so it barely tilts the line. The red point sits far out in x and off the trend, so it drags the whole fit toward itself. The standard summary is Cook's distance, which multiplies leverage by discrepancy, one number per observation flagging the points worth investigating. When you find one, investigate, do not just delete: a genuine data-entry error can be removed, but a real extreme value may be the most important row you have. Fitting the model with and without it, and reporting both, is the honest move.
Fixing Violations: the Remedy Catalog
Diagnosis is only useful if you know the cure. Match the symptom to the fix:
| Symptom | Likely cause | Remedy |
|---|---|---|
| Curve in residuals | missing nonlinearity | add x² or interaction, transform x, splines (see Regularization & Flexible Models) |
| Funnel (growing spread) | variance scales with the mean | model log(y); weighted least squares; robust (HC) SEs |
| Skewed / heavy-tailed residuals | skewed response | log or Box-Cox transform of y |
| Residuals track over time | autocorrelation | Newey-West SEs, add lags, time-series models (see Econometrics & Panel Data) |
| A few points dominate | influential outliers | investigate; robust regression; report with and without |
| Coefficients unstable | multicollinearity | drop/combine predictors; ridge (see Regularization & Flexible Models) |
The single most common and most effective remedy is the log transform of a skewed, positive response. It routinely fixes curvature, funnel, and skew in one move, as the next section shows on real data.
Real-World Example: Diagnosing an Insurance Model
We model annual medical charges from age, BMI, and smoking. The naive fit looks fine on R², then
the diagnostics expose two violations, and one transform fixes both.
One row per policyholder with age, bmi,
smoker (yes / no), and annual charges (dollars).
Step 1, check the data. Before modeling, look at the response. Annual charges are heavily right-skewed, a few very expensive policies stretch out a long tail:
That shape is a warning. A right-skewed, strictly positive response whose spread grows with its level is exactly what makes a raw-dollar model's residuals fan out, so we should not be surprised if the equal-variance and normality checks fail below.
Steps 2–3, fit and evaluate. We fit charges ~ age + bmi + C(smoker). In raw dollars it
posts a respectable R² = 0.70, which looks fine on its own.
Step 4, check the conditions. Now the residuals tell the truth. They fail two of the LINE checks, and
modeling log(charges) instead repairs both while even fitting better:
| Check | charges (levels) | log(charges) |
|---|---|---|
| Equal variance (Breusch-Pagan p) | <0.001 ✗ funnel | 0.88 ✓ flat |
| Normality (Jarque-Bera p) | <0.001 ✗ right-skewed | 0.50 ✓ straight Q-Q |
| Independence (Durbin-Watson) | 2.17 ✓ | 1.94 ✓ |
| R² | 0.70 | 0.78 |
The picture is the whole argument: in dollars (left) the residuals fan out and fail Breusch-Pagan; on
log(charges) (right) they settle into a flat, healthy band that passes. The remedy from the summary
table above, applied to real data.
Step 5, interpret and predict. On the log scale the coefficients read as percentages: each year of age adds about 3.2% to charges, each BMI point about 3.1%, and being a smoker multiplies charges by e1.47 ≈ 4.3×, the single biggest driver. Cook's distance flags a handful of high-charge policyholders (max ≈ 0.03) worth a look, but none overturns the story. The naive levels model would have reported the same smoker effect with untrustworthy standard errors; the diagnostics are what let you believe the p-values.
Predict. Because we modeled the log, predict on the log scale, then exponentiate back to dollars:
A 45-year-old non-smoker with BMI 30: exp(6.62 + 1.46 + 0.92) = exp(9.00) ≈ $8,100. Skip the exponential and you would report a meaningless "log-dollar" number, the one trap of modeling on a transformed scale.
Diagnostics in Machine Learning & AI
Model checking does not disappear in ML; it changes name and grows more important as models grow more opaque.
| Idea (this chapter) | In ML / AI it becomes | Example |
|---|---|---|
| Residual analysis | Error analysis on held-out data | plotting where predictions fail, by segment |
| Heteroscedastic errors | Non-uniform uncertainty | quantile regression, heteroscedastic likelihoods |
| Influential points | Influence functions & data valuation | which training points most change the model |
| Normality of errors | Calibration of predicted distributions | calibration plots, proper scoring rules |
| Assumption checks | Subgroup / fairness audits | catching systematic error on a subpopulation |
A single validation score is the machine-learning equivalent of reporting R² and stopping, it can be high while the model is systematically wrong on a subgroup that the aggregate number averages away. The discipline of this chapter, look at the residuals, by segment, not just the summary, is exactly what modern error analysis and fairness auditing formalize. Influence functions, now used to trace a model's behavior back to individual training examples, are Cook's distance grown up. And because deep models rarely emit honest error bars by default, the heteroscedasticity and normality checks here reappear as calibration: does the model's stated confidence match its real accuracy? The habits scale; the summary statistic never tells the whole story.
Run the full diagnostic suite
The companion notebook builds the classic four-plot diagnostic dashboard with statsmodels,
runs Breusch-Pagan, Jarque-Bera, and Durbin-Watson, computes leverage and Cook's distance,
then applies the log transform to regression-assumptions-and-diagnostics--insurance.xlsx and watches every violation clear, before
and after, side by side.
View opens the rendered notebook instantly (no setup). Open in Colab runs &
edits it live in your browser. To run locally, install numpy, pandas, scipy,
matplotlib, seaborn, statsmodels, and openpyxl and launch
jupyter notebook.
🎓 Key Takeaways
- ✓Trust rests on the LINE assumptions: Linearity, Independence, Normality, Equal variance, checked with residual and Q-Q plots plus formal tests.
- ✓Learn the shapes: a random band is healthy; a curve, a funnel, or a bent Q-Q plot each name a specific violation and its fix.
- ✓Influence = leverage × discrepancy: use Cook's distance to flag points that bend the fit, then investigate rather than delete.
- ✓Most violations have a simple cure, and a log transform of a skewed positive response fixes curvature, funnel, and skew at once.
- ✓Real data: the insurance model failed Breusch-Pagan and Jarque-Bera in dollars but passed both in log(charges), which even raised R² from 0.70 to 0.78.
Practice Challenges
Five short challenges using the statsmodels diagnostic tools.
Build the diagnostic dashboard
Fit charges ~ age + bmi + C(smoker) and produce residual-vs-fitted, Q-Q, and scale-location plots.
model.resid, model.fittedvalues, sm.qqplot.Test every assumption
Run Breusch-Pagan, Jarque-Bera, and Durbin-Watson; state which LINE conditions hold.
het_breuschpagan, jarque_bera, durbin_watson.Fix it with a log
Refit on log(charges) and show the funnel and skew are gone.
Find influential points
Compute leverage, studentized residuals, and Cook's distance; flag the top few observations.
model.get_influence() gives cooks_distance and hat_matrix_diag.Refit without the influencers
Drop the highest-Cook's-distance points, refit, and compare the coefficients, are the conclusions stable?
A fully-worked solutions notebook walks through all five challenges, each verified in code. Try them yourself first, then compare.
Quiz: Test Yourself
Eight quick questions on assumptions and diagnostics. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.
You can now certify a linear model as sound, or diagnose and repair it. Logistic Regression carries the whole workflow to a new kind of outcome, a yes/no response, where the line becomes an S-curve.