Chapter 44 — Conditional Expectation Challenge Solutions

Chapter 44 · Solutions

Conditional Expectation, Worked Answers ✅

Full solutions to the five challenges. Try them yourself first, then compare.

Statistics, Data Science and AI: A Visual Handbook · John Fisher · 2026

⚙️ Setup¶

In [1]:

import numpy as np
rng = np.random.default_rng(441)
print("ready")

ready

CHALLENGE 1 · CONDITIONAL MEAN

Average within a group

A factory has two machines. Machine A makes parts with mean weight 100g (sd 5); machine B mean 110g (sd 5). Simulate and estimate E[weight | machine].

In [2]:

m = rng.integers(0,2,size=40_000)
w = rng.normal(np.where(m==0,100,110), 5)
print(f"E[weight | A] = {w[m==0].mean():.2f}  (true 100)")
print(f"E[weight | B] = {w[m==1].mean():.2f}  (true 110)")

E[weight | A] = 100.01  (true 100)
E[weight | B] = 110.01  (true 110)

Answer: the conditional expectation is about 100g given machine A and 110g given machine B. E[Y | X] is a function of X: one average per machine, not a single overall number.

CHALLENGE 2 · TOWER PROPERTY

Recover the overall mean

Using the same data, verify E[E[weight|machine]] = E[weight] by averaging the two conditional means, weighted by how often each machine is used.

In [3]:

w_A, w_B = (m==0).mean(), (m==1).mean()
tower = w_A*w[m==0].mean() + w_B*w[m==1].mean()
print(f"E[weight] directly = {w.mean():.3f}")
print(f"tower E[E[W|M]]    = {tower:.3f}")

E[weight] directly = 104.985
tower E[E[W|M]]    = 104.985

Answer: both equal about 105g. The tower property recovers the overall mean by averaging the within-machine means, weighted by usage. Because each machine runs about half the time, the overall mean sits near the midpoint of 100 and 110.

CHALLENGE 3 · BEST PREDICTOR

Conditional mean beats the global mean

Show that predicting each part's weight with its machine's conditional mean gives a lower mean squared error than always predicting the overall mean.

In [4]:

pred_cond = np.where(m==0, w[m==0].mean(), w[m==1].mean())
pred_glob = np.full_like(w, w.mean())
print(f"MSE with E[W|M]    = {((w-pred_cond)**2).mean():.2f}")
print(f"MSE with E[W] only = {((w-pred_glob)**2).mean():.2f}")

MSE with E[W|M]    = 25.04
MSE with E[W] only = 50.04

Answer: conditioning on the machine lowers the MSE (about 25, the within-group variance 5²) versus ignoring it (about 50). No function of the machine label can beat E[W | machine] on squared error, which is exactly why it is "the best predictor".

CHALLENGE 4 · TOTAL VARIANCE

Within plus between

Decompose the total variance of weight into E[Var(W|M)] (within) and Var(E[W|M]) (between), and check they sum to Var(W).

In [5]:

wts = np.array([(m==0).mean(),(m==1).mean()])
cm  = np.array([w[m==0].mean(), w[m==1].mean()])
within  = sum(wts[k]*w[m==k].var() for k in range(2))
between = (wts*(cm - w.mean())**2).sum()
print(f"Var(W) total = {w.var():.2f}")
print(f"within {within:.2f} + between {between:.2f} = {within+between:.2f}")

Var(W) total = 50.04
within 25.04 + between 25.00 = 50.04

Answer: the within-machine variance (≈ 25, from the sd of 5) plus the between-machine variance (≈ 25, from the 10g gap split across two equal groups) sums to the total variance of weight. This law of total variance underlies ANOVA and bias-variance analysis.

CHALLENGE 5 · REGRESSION AS E[Y|X]

Recover a curve

Generate y = x^2 + noise and estimate E[Y | X] by binning. Confirm the estimated conditional mean near x=2 is close to 4.

In [6]:

x = rng.uniform(0,3,size=5000); y = x**2 + rng.normal(0,0.5,size=5000)
near2 = (x>1.9)&(x<2.1)
print(f"estimated E[Y | X near 2] = {y[near2].mean():.3f}  (true 2^2 = 4)")

estimated E[Y | X near 2] = 4.047  (true 2^2 = 4)

Answer: the estimated conditional mean near x = 2 is about 4.0, matching the true E[Y | X] = x². Regression of any kind, linear, tree-based, or neural, is an estimator of this conditional-expectation function.

🎉 Nicely done

You computed conditional means, verified the tower property, showed E[Y|X] minimizes squared error, decomposed variance, and recovered a regression curve. E[Y|X] is the bridge from probability to prediction.