Chapter 64 · Solutions
Non-Probability Sampling — Worked Solutions ✅
Five challenges, each verified in code.
⚙️ Setup & population¶
In [1]:
import numpy as np
rng = np.random.default_rng(630)
N=200_000
age = rng.integers(18,78,N)
value = np.clip(rng.normal(20-0.22*(age-18),6),0,None) # younger -> higher value
TRUE = value.mean(); print(f"true mean = {TRUE:.2f}")
true mean = 13.59
CHALLENGE 1
Convenience bias is constant
Sample only the young, reachable half of the population at n=100, 1,000, 10,000 and show the bias does not shrink.
In [2]:
pool = np.where(age < 40)[0]
for n in [100,1000,10000]:
est = value[rng.choice(pool,n,replace=False)].mean()
print(f"n={n:>6}: estimate {est:.2f}, bias {est-TRUE:+.2f}")
print("systematic bias: same offset at every n")
n= 100: estimate 18.10, bias +4.51 n= 1000: estimate 17.79, bias +4.20 n= 10000: estimate 17.61, bias +4.01 systematic bias: same offset at every n
CHALLENGE 2
Voluntary response over-samples extremes
On a 0-10 satisfaction scale, let only the extremes respond. Compare the extreme share in the population to the responders.
In [3]:
sat = np.clip(rng.normal(6.5,1.8,N),0,10)
p = 0.02 + 0.5*((sat-5)/5)**2
vol = sat[rng.random(N)<p]
ex = lambda x: np.mean((x<=2)|(x>=9))*100
print(f"population extreme share: {ex(sat):.0f}%")
print(f"volunteer extreme share: {ex(vol):.0f}%")
print(f"population mean {sat.mean():.2f} vs volunteer mean {vol.mean():.2f}")
population extreme share: 9% volunteer extreme share: 31% population mean 6.48 vs volunteer mean 7.82
CHALLENGE 3
Quota reduces but does not remove bias
Match age-band proportions but pick high-value people within each band. Show the bias is smaller than pure convenience but not zero.
In [4]:
bands = np.digitize(age,[30,45,60])
shares = np.array([np.mean(bands==b) for b in range(4)])
picks=[]
for b in range(4):
idx=np.where(bands==b)[0]; w=value[idx]+1
picks.append(value[rng.choice(idx, round(4000*shares[b]), replace=False, p=w/w.sum())])
q=np.concatenate(picks)
conv = value[rng.choice(np.where(age<40)[0],4000,replace=False)]
print(f"quota bias = {q.mean()-TRUE:+.2f}")
print(f"convenience bias = {conv.mean()-TRUE:+.2f}")
print("quota helps, residual bias remains inside the quotas")
quota bias = +2.56 convenience bias = +4.15 quota helps, residual bias remains inside the quotas
CHALLENGE 4
Snowball over-samples the well-connected
With value rising in social connectedness, sample with probability proportional to connections and show the upward bias.
In [5]:
conn = rng.integers(0,50,N)
val2 = np.clip(rng.normal(14+0.18*conn,5),0,None)
snow = val2[rng.choice(N,3000,replace=False,p=conn/conn.sum())]
print(f"true mean {val2.mean():.2f}, snowball estimate {snow.mean():.2f}, bias {snow.mean()-val2.mean():+.2f}")
true mean 18.41, snowball estimate 19.83, bias +1.42
CHALLENGE 5
Fixing it with post-stratification weights
Reweight a convenience sample so each age band counts according to its TRUE population share, and recover the mean.
In [6]:
pool = np.where(rng.random(N) < 1/(1+np.exp(0.12*(age-35))))[0] # reachable skew young
samp = rng.choice(pool, 5000, replace=False)
sb = np.digitize(age[samp],[30,45,60])
pop_share = np.array([np.mean(np.digitize(age,[30,45,60])==b) for b in range(4)])
samp_share = np.array([np.mean(sb==b) for b in range(4)])
w = (pop_share/samp_share)[sb] # up-weight under-represented bands
naive = value[samp].mean()
weighted = np.average(value[samp], weights=w)
print(f"true mean = {TRUE:.2f}")
print(f"naive estimate = {naive:.2f} (bias {naive-TRUE:+.2f})")
print(f"weighted estimate= {weighted:.2f} (bias {weighted-TRUE:+.2f})")
print("post-stratification weights correct KNOWN imbalances")
true mean = 13.59 naive estimate = 17.03 (bias +3.43) weighted estimate= 13.99 (bias +0.40) post-stratification weights correct KNOWN imbalances
Statistics, Data Science and AI: A Visual Handbook · © 2026 John Fisher