⚙️ Setup¶
import numpy as np
from scipy import stats
rng = np.random.default_rng(451)
print("ready")
ready
X = rng.poisson(3, size=100_000)
print(f"mean = {X.mean():.3f} (true 3), var = {X.var():.3f} (true 3)")
print(f"skew = {stats.skew(X):.3f} (true 1/sqrt(3)={1/np.sqrt(3):.3f}), kurtosis = {stats.kurtosis(X):.3f} (true 1/3)")
mean = 2.997 (true 3), var = 3.028 (true 3) skew = 0.604 (true 1/sqrt(3)=0.577), kurtosis = 0.369 (true 1/3)
Answer: for Poisson(λ=3), mean = variance = 3 (its signature), skewness = 1/√λ ≈ 0.577, and excess kurtosis = 1/λ ≈ 0.333. The moments match the known Poisson formulas.
X = rng.normal(5, 2, size=500_000)
M = lambda t: np.mean(np.exp(t*X)); h=1e-3
mean_mgf = (M(h)-M(-h))/(2*h)
print(f"M'(0) = {mean_mgf:.3f} (true mean 5)")
M'(0) = 4.996 (true mean 5)
Answer: the numerical derivative M′(0) ≈ 5, the mean. The normal MGF exp(μt + σ²t²/2) differentiates at 0 to μ, and the empirical estimate confirms it.
# 54 to 86 is mu +/- 2 sd; Chebyshev: P(|X-mu| >= k sd) <= 1/k^2
k = 2
print(f"Chebyshev: P(outside mu +/- {k} sd) <= 1/{k}^2 = {1/k**2:.2f}")
print("So at most 25% of scores fall outside 54-86, whatever the shape.")
Chebyshev: P(outside mu +/- 2 sd) <= 1/2^2 = 0.25 So at most 25% of scores fall outside 54-86, whatever the shape.
Answer: 54 and 86 are μ ± 2σ, so Chebyshev guarantees at most 1/2² = 25% lie outside, hence at least 75% inside, for any distribution. (If it were normal, the real figure would be about 95% inside, Chebyshev is a worst-case bound.)
mean, a = 200, 1000
print(f"Markov: P(X >= {a}) <= E[X]/a = {mean}/{a} = {mean/a:.2f}")
Markov: P(X >= 1000) <= E[X]/a = 200/1000 = 0.20
Answer: Markov gives P(X ≥ 1000) ≤ 200/1000 = 0.20. With nothing but the mean, we can still cap the chance of a slow response at 20%. Markov is crude but assumption-free, it needs only that X ≥ 0 and a known mean.
eps, delta = 0.03, 0.01
n = int(np.ceil(np.log(2/delta)/(2*eps**2)))
print(f"n >= ln(2/{delta})/(2*{eps}^2) = {n:,} examples")
n >= ln(2/0.01)/(2*0.03^2) = 2,944 examples
Answer: n ≥ ln(2/0.01)/(2·0.03²) ≈ 2,944 examples. Hoeffding turns a desired precision and confidence into a concrete sample size, the kind of guarantee that underlies PAC learning and honest model evaluation.