Do advertising spend and sales rise together? Does resting heart rate fall as exercise rises? These are questions about association, how two variables co-vary. The simplest measure is covariance, and it is the foundation everything in this Part builds on.
Covariance reads the direction two variables move together from its sign. But its magnitude depends on the units, which is why we standardize it into correlation.
What Covariance Measures
Picture a scatterplot. If high x tends to come with high y (and low with low), the cloud slopes up and covariance is positive. If high x comes with low y, it slopes down and covariance is negative. A shapeless cloud gives covariance near zero.
In the notebook these three pictures come straight from np.cov: covariance about +0.8, −0.8, and
0.0. One signed number captures "which way do they move together?", the starting point for every relationship
measure that follows.
The Formula & the Quadrant Intuition
Split the scatterplot into four quadrants with vertical and horizontal lines at the two means. A point where both x and y are above their means (upper-right) or both below (lower-left) has a positive deviation product. The other two quadrants give negative products. Covariance is the average of all of them.
It is a vote: each point casts a signed product, and the average is the covariance. In the notebook, coloring points
by the sign of their product makes a positive relationship visibly purple-dominated. (Use ddof to pick
the population or sample version; the sign and story are the same.)
Covariance vs Correlation
Covariance has a fatal flaw for comparison: its units are x times y, so its magnitude is arbitrary. Measure y in cents instead of dollars and the covariance jumps 100-fold, though nothing about the relationship changed. Divide by the two standard deviations and the units cancel, giving correlation, a unitless number locked into [−1, 1].
So the two are partners. Covariance gives the raw signed strength in the data's own units; correlation rescales it to a universal −1-to-1 ruler you can compare across any pair of variables. Correlation Coefficients is devoted to that ruler.
Real-World Example: Ad Spend & Sales
A marketing team has 160 campaigns with spend, reach, visits, and sales. We compute the covariance matrix (a grid of unit-dependent numbers) and the correlation matrix (a readable heatmap), and see why analysts almost always reach for the latter.
One row per campaign with ad_spend, impressions,
web_visits, and sales (plus region).
| Pair | Covariance | Correlation | Reading |
|---|---|---|---|
| ad_spend & sales | ≈ 29,000,000 | 0.95 | very strong, positive |
| ad_spend & web_visits | large positive | 0.97 | very strong, positive |
| web_visits & sales | large positive | 0.91 | strong, positive |
The covariance between spend and sales is roughly 29 million (dollars times dollars), a magnitude
you cannot interpret or compare to anything. Standardized, that same relationship is a correlation of
0.95, instantly legible as "very strong and positive." Every metric here moves together, which is
precisely why analysts read the correlation heatmap, not the covariance grid. The notebook
draws both with pandas and seaborn.
How to read it: each cell is the correlation between its row and column variable, so the grid is symmetric and the diagonal is always 1.00 (a variable correlates perfectly with itself). Darker purple means a stronger positive correlation. Here every off-diagonal cell is dark (0.91 to 0.99): all four marketing metrics rise together almost in lockstep, more spend buys more impressions and visits, which turn into more sales. Why it matters: a heatmap turns a wall of numbers into an instant picture, you spot the strongest and weakest links at a glance, which is exactly why the correlation heatmap is the standard first look at any set of numeric variables.
Covariance in Machine Learning & AI
The covariance matrix is one of the most important objects in all of machine learning.
| Idea (this chapter) | In ML / AI it becomes | Example |
|---|---|---|
| Covariance matrix | The shape of multivariate data | the Σ in a multivariate Gaussian |
| Directions of variance | Principal Component Analysis (PCA) | eigenvectors of the covariance matrix |
| Standardizing | Whitening & Mahalanobis distance | distance that accounts for correlation |
| Sign of covariance | Feature relationship screening | which inputs move with the target |
The covariance matrix encodes how every pair of features varies together, and its eigen-decomposition is exactly PCA: the eigenvectors point along the directions of greatest variance, letting you compress or denoise high-dimensional data. The same matrix defines the multivariate Gaussian at the heart of many models, and Mahalanobis distance uses its inverse to measure distance in a way that respects correlation (crucial for anomaly detection). Covariance is where one-variable statistics ends and the multivariate world of modern ML begins.
See covariance in pictures with Python
The companion notebook plots positive, negative, and zero covariance, colors the deviation quadrants, shows
how units change covariance but not correlation, and loads covariance--ad_sales.xlsx to compare the
covariance matrix with a seaborn correlation heatmap.
View opens the rendered notebook instantly (no setup). Open in Colab runs &
edits it live in your browser. To run locally, install numpy, pandas, scipy,
matplotlib, seaborn, statsmodels, and openpyxl and launch jupyter notebook.
🎓 Key Takeaways
- ✓Covariance = mean of (x − x̄)(y − ȳ); positive = move together, negative = move oppositely, ~0 = no linear tendency.
- ✓Quadrant intuition: points on the same side of both means push covariance up; opposite-side points push it down.
- ✓Units matter: covariance scales with the units of x and y, so its magnitude is not comparable across pairs.
- ✓Standardize by the two SDs to get correlation, unitless and in [−1, 1] (next chapter).
- ✓In ML/AI: the covariance matrix drives PCA, the multivariate Gaussian, and Mahalanobis distance.
Practice Challenges
Five short challenges, beginner to intermediate. Try them with NumPy and pandas before checking the solutions.
Covariance from the formula
For x = [1,2,3,4,5], y = [2,4,5,4,5], compute covariance as the mean of deviation products and check with NumPy.
((x-x.mean())*(y-y.mean())).mean() matches np.cov(ddof=0).Sign of covariance
Generate a negatively related pair and confirm the covariance is negative.
Units change covariance, not correlation
Show cov(x, 100y) = 100·cov(x, y) while the correlation is unchanged.
np.cov and np.corrcoef.Standardize to correlation
Compute correlation as cov/(sₓ·sₐ) and confirm it matches np.corrcoef.
ddof=1 standard deviations.Real data: covariance vs correlation matrix
Load covariance--ad_sales.xlsx and print the covariance and correlation matrices of the four numeric columns.
df.cov() and df.corr().A fully-worked solutions notebook walks through all five challenges, each verified in code. Try them yourself first, then compare.
Quiz: Test Yourself
Eight quick questions on covariance. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.
Covariance told us the direction; its magnitude was unreadable. Correlation Coefficients standardizes it into Pearson's r and Spearman's rank correlation, the −1-to-1 ruler for the strength of a relationship.