⚙️ Setup¶
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
rng = np.random.default_rng(166)
INK="#1a2138"; CYAN="#0891b2"; PURPLE="#7c3aed"; AMBER="#d97706"; GREEN="#059669"; PINK="#db2777"; BLUE="#2563eb"; GRID="#e6e9f2"
plt.rcParams.update({"figure.facecolor":"white","axes.facecolor":"white","figure.dpi":110,"font.size":11,
"axes.edgecolor":GRID,"axes.grid":True,"grid.color":GRID,"axes.axisbelow":True,"axes.spines.top":False,
"axes.spines.right":False,"axes.titlesize":12,"axes.titleweight":"bold","legend.frameon":False})
print("Ready.")
Ready.
n=300; a=rng.normal(0,1,n); b=a*0.98+rng.normal(0,0.2,n) # b ~ a (redundant)
c=rng.normal(0,1,n); d=-0.5*c+rng.normal(0,0.9,n)
data=pd.DataFrame({"a":a,"b":b,"c":c,"d":d}); corr=data.corr().values; cols=list(data.columns)
fig,ax=plt.subplots(figsize=(5.2,4.6))
im=ax.imshow(corr,cmap="RdBu_r",vmin=-1,vmax=1)
ax.set_xticks(range(4)); ax.set_xticklabels(cols); ax.set_yticks(range(4)); ax.set_yticklabels(cols); ax.grid(False)
for i in range(4):
for j in range(4):
ax.text(j,i,f"{corr[i,j]:.2f}",ha="center",va="center",color="white" if abs(corr[i,j])>0.55 else INK,fontweight="bold")
fig.colorbar(im,ax=ax,shrink=0.8); plt.tight_layout(); plt.show()
Answer: Variables a and b are the most correlated (~0.98), so they are nearly redundant; in modeling you would drop or combine one. A heatmap shows association, not causation: a high value only says the two move together, never that one drives the other. It also captures only linear (Pearson) correlation, so a strong curved relationship can read near 0.
n=150
g0=pd.DataFrame({"x":rng.normal(2,0.6,n),"y":rng.normal(2,0.6,n),"z":rng.normal(5,1.2,n),"grp":"A"})
g1=pd.DataFrame({"x":rng.normal(4,0.6,n),"y":rng.normal(4,0.6,n),"z":rng.normal(5,1.2,n),"grp":"B"})
data=pd.concat([g0,g1],ignore_index=True)
colors=data["grp"].map({"A":CYAN,"B":AMBER})
axes=pd.plotting.scatter_matrix(data[["x","y","z"]],figsize=(7.5,7.5),diagonal="hist",color=colors,s=14,alpha=0.8)
for ax in axes.ravel(): ax.grid(False)
plt.suptitle("SPLOM (color = group)",y=1.0,fontweight="bold"); plt.tight_layout(); plt.show()
Answer: The x vs y panel separates the two groups best (their clouds barely overlap), while any pair involving z mixes them, because z has the same distribution in both groups. The diagonal shows each variable's own distribution (here a histogram), which is why you cannot scatter a variable against itself.
income=np.array([10,20,30,40,100.0]); x=np.sort(income); n=len(x)
cum=np.cumsum(x)/x.sum()
ly=np.concatenate([[0],cum]); lx=np.linspace(0,1,n+1)
area_under=np.sum((lx[1:]-lx[:-1])*(ly[1:]+ly[:-1])/2)
gini=1-2*area_under
fig,ax=plt.subplots(figsize=(5.6,5.2))
ax.plot([0,1],[0,1],ls="--",color=INK,label="equality")
ax.plot(lx,ly,color=PINK,lw=2.4,marker="o",label=f"Lorenz (Gini={gini:.2f})")
ax.fill_between(lx,ly,lx,color=PINK,alpha=0.12)
ax.set_xlabel("cumulative population"); ax.set_ylabel("cumulative income"); ax.legend(loc="upper left")
plt.tight_layout(); plt.show()
print(f"Gini = {gini:.3f}")
Gini = 0.400
Answer: The one large income (100) bows the curve well below the diagonal, giving a Gini of about 0.37. The recipe: sort, take cumulative income shares for the y-values and equal population steps for x, then Gini = 1 − 2·(area under the Lorenz curve). 0 would be perfect equality; 1 would be one person holding everything.
print("(a) correlation -1..1 -> DIVERGING (centered at 0), e.g. RdBu_r")
print("(b) rainfall totals -> SEQUENTIAL (one-way magnitude), e.g. viridis/Blues")
print("(c) percent change vs 0 -> DIVERGING (meaningful midpoint at 0)")
(a) correlation -1..1 -> DIVERGING (centered at 0), e.g. RdBu_r (b) rainfall totals -> SEQUENTIAL (one-way magnitude), e.g. viridis/Blues (c) percent change vs 0 -> DIVERGING (meaningful midpoint at 0)
Answer: (a) diverging, centered at 0, since +1 and −1 are opposite; (b) sequential, since rainfall only runs low to high with no meaningful middle; (c) diverging, because 0% change is a real midpoint separating gains from losses. Avoid rainbow/jet: it is perceptually non-uniform, so equal data steps look unequal, it invents false bands, and it fails in grayscale and for colorblind readers. Prefer perceptually uniform maps like viridis.
v1, v2 = 10, 40 # v2 is 4x v1
# correct: area proportional to value -> radius proportional to sqrt(value)
r_ratio_correct = np.sqrt(v2/v1)
# wrong: radius proportional to value -> area proportional to value^2
area_ratio_wrong = (v2/v1)**2
print(f"value ratio = {v2/v1:.0f}x")
print(f"correct radius ratio = sqrt(4) = {r_ratio_correct:.0f}x (area is then 4x, matching the value)")
print(f"wrong (radius=value) -> area looks {area_ratio_wrong:.0f}x bigger, a 4x value reads as 16x")
value ratio = 4x correct radius ratio = sqrt(4) = 2x (area is then 4x, matching the value) wrong (radius=value) -> area looks 16x bigger, a 4x value reads as 16x
Answer: For a 4x value the radius should grow by √4 = 2x, so the area grows 4x and matches the value. If you instead make the radius proportional to the value (2x value to 2x radius... here 4x radius), the area scales with the square, so a 4x value balloons to look 16x bigger. In matplotlib the scatter s argument is already area, so pass the value (or a constant times it) straight to s.