←Back to the bookChapter 86 · Case Study: A Clinical Trial

Chapter 86 · Case Study · Inference

Case Study: A Clinical Trial 🩺

One randomized trial, three questions: did cholesterol fall on the drug, did it fall more than placebo, and were more patients responders? We check baseline balance first, then use scipy and statsmodels for the three tests and their intervals.

Statistics, Data Science and AI: A Visual Handbook · John Fisher · 2026

⚙️ Setup

import numpy as np, pandas as pd, matplotlib.pyplot as plt
from scipy import stats
# statsmodels = the R/SAS-style stats library (pre-installed on Colab): it does the standard errors,
# test statistics, intervals, and post-hoc comparisons for us, so we write far less by-hand code.
import statsmodels.api as sm
from statsmodels.formula.api import ols
from statsmodels.stats.proportion import proportions_ztest, confint_proportions_2indep, proportion_confint
from statsmodels.stats.multicomp import pairwise_tukeyhsd
from statsmodels.stats.weightstats import DescrStatsW, CompareMeans
from scipy.stats.contingency import association   # Cramer's V in one call
CY="#0891b2"; DEEP="#0e7490"; LIGHT="#67e8f9"; INK="#1a2138"; GRID="#e6e9f2"; GREEN="#059669"; RED="#ef4444"; AMBER="#d97706"
plt.rcParams.update({"figure.facecolor":"white","axes.facecolor":"white","figure.dpi":110,"font.size":11,
   "axes.edgecolor":GRID,"axes.grid":True,"grid.color":GRID,"axes.axisbelow":True,"axes.spines.top":False,
   "axes.spines.right":False,"axes.titlesize":12,"axes.titleweight":"bold","legend.frameon":False})
pd.set_option("display.width",120)
BASE="https://raw.githubusercontent.com/johnfisher-ai/Statistics-Data-Science-AI-Visual-Book/main/data/"
rng = np.random.default_rng(86)

STEP 1 · EXPLORE THE DATA

Summaries, and a baseline-balance check

The first thing a trial statistician checks is whether randomization balanced the arms at BASELINE: if the treatment group started healthier (lower LDL or younger), any 'effect' could be that head start. We summarize the data and compare the arms before treatment.

try:    d = pd.read_excel("../../data/case-study-a-clinical-trial--clinical_trial.xlsx", sheet_name="Patients")
except FileNotFoundError: d = pd.read_excel(BASE+"case-study-a-clinical-trial--clinical_trial.xlsx", sheet_name="Patients")
d["change"] = d.ldl_before - d.ldl_after   # positive = LDL dropped
print("shape:", d.shape, "| missing:", d.isna().sum().sum())
print(d.groupby("arm")[["age","ldl_before","ldl_after","change","responder"]].mean().round(1))

shape: (170, 7) | missing: 0
            age  ldl_before  ldl_after  change  responder
arm                                                      
placebo    58.4       170.1      159.8    10.3        0.2
treatment  57.1       170.9      142.2    28.6        0.5

# baseline balance: arms should match on age and starting LDL (a quick t-test on baselines)
tr0=d[d.arm=="treatment"]; pl0=d[d.arm=="placebo"]
print(f"baseline age:  treat {tr0.age.mean():.1f} vs placebo {pl0.age.mean():.1f}  (t-test p={stats.ttest_ind(tr0.age,pl0.age).pvalue:.2f})")
print(f"baseline LDL:  treat {tr0.ldl_before.mean():.1f} vs placebo {pl0.ldl_before.mean():.1f}  (t-test p={stats.ttest_ind(tr0.ldl_before,pl0.ldl_before).pvalue:.2f})")
print("-> baselines are balanced (large p-values), so randomization worked; differences at week 12 are about the drug")

baseline age:  treat 57.1 vs placebo 58.4  (t-test p=0.43)
baseline LDL:  treat 170.9 vs placebo 170.1  (t-test p=0.81)
-> baselines are balanced (large p-values), so randomization worked; differences at week 12 are about the drug

No missing data, and the arms start out the same on age and baseline LDL (balance confirmed), so we can trust that week-12 differences reflect the treatment, not a head start. Now the three questions.

STEP 2 · THREE QUESTIONS, THREE TESTS

Match each question to its design

Q1 is PAIRED (same patient, before vs after) -> paired t-test. Q2 compares the CHANGE between two independent arms -> two-sample (Welch) t-test. Q3 compares a RESPONDER RATE (yes/no) -> two-proportion z-test.

print("Q1 within treatment, before vs after  -> PAIRED t-test     | H0: no change   H1: LDL decreased")
print("Q2 LDL change, treatment vs placebo    -> TWO-SAMPLE Welch t | H0: equal change H1: treatment larger")
print("Q3 responder rate, treatment vs placebo-> TWO-PROPORTION z   | H0: pT = pP      H1: pT > pP")

Q1 within treatment, before vs after  -> PAIRED t-test     | H0: no change   H1: LDL decreased
Q2 LDL change, treatment vs placebo    -> TWO-SAMPLE Welch t | H0: equal change H1: treatment larger
Q3 responder rate, treatment vs placebo-> TWO-PROPORTION z   | H0: pT = pP      H1: pT > pP

STEP 3 · RUN THE ANALYSIS

statsmodels for the intervals and the proportion test

scipy gives the paired and two-sample t one-liners; statsmodels supplies the matching confidence intervals (CompareMeans, DescrStatsW) and the whole two-proportion test + interval, so we never hand-roll a standard error.

tr=d[d.arm=="treatment"]; pl=d[d.arm=="placebo"]
# Q1 paired t-test, with the CI on the mean drop from statsmodels
q1=stats.ttest_rel(tr.ldl_after, tr.ldl_before)
ci1=DescrStatsW(tr.change.values).tconfint_mean()
print(f"Q1 PAIRED drop = {tr.change.mean():.1f} mg/dL, t={q1.statistic:.2f}, p={q1.pvalue:.2e}, 95% CI {tuple(round(float(v),1) for v in ci1)}")
# Q2 two-sample Welch t, with CI for the difference from statsmodels CompareMeans
cm=CompareMeans(DescrStatsW(tr.change.values), DescrStatsW(pl.change.values))
q2=stats.ttest_ind(tr.change, pl.change, equal_var=False)
ci2=cm.tconfint_diff(usevar="unequal")
print(f"Q2 extra drop = {tr.change.mean()-pl.change.mean():.1f} mg/dL (treat {tr.change.mean():.1f} vs placebo {pl.change.mean():.1f})")
print(f"   Welch t={q2.statistic:.2f}, p={q2.pvalue:.2e}, 95% CI for extra drop {tuple(round(float(v),1) for v in ci2)}")

Q1 PAIRED drop = 28.6 mg/dL, t=-19.65, p=9.42e-34, 95% CI (25.7, 31.5)
Q2 extra drop = 18.3 mg/dL (treat 28.6 vs placebo 10.3)
   Welch t=8.45, p=1.45e-14, 95% CI for extra drop (14.0, 22.6)

# Q3 responder rate: statsmodels two-proportion test + interval
sT,nT=tr.responder.sum(),len(tr); sP,nP=pl.responder.sum(),len(pl)
z,p3=proportions_ztest([sT,sP],[nT,nP],alternative="larger")
lo,hi=confint_proportions_2indep(sT,nT,sP,nP,method="wald")
print(f"Q3 responders: treatment {sT/nT:.1%} vs placebo {sP/nP:.1%}; z={z:.2f}, one-sided p={p3:.2e}")
print(f"   95% CI for the gap: [{lo*100:+.1f}, {hi*100:+.1f}] pts")
fig,ax=plt.subplots(1,2,figsize=(11,3.5))
ax[0].boxplot([pl.change.values, tr.change.values], tick_labels=["placebo","treatment"], patch_artist=True, boxprops=dict(facecolor=LIGHT,alpha=0.6))
ax[0].axhline(0,color=INK,lw=1,ls=":"); ax[0].set_ylabel("LDL drop (mg/dL)"); ax[0].set_title("LDL reduction by arm")
ax[1].bar(["placebo","treatment"],[sP/nP,sT/nT],color=[LIGHT,CY],width=0.55)
for i,v in enumerate([sP/nP,sT/nT]): ax[1].text(i,v+0.02,f"{v:.0%}",ha="center",fontweight="bold")
ax[1].set_ylim(0,0.75); ax[1].set_ylabel("responder rate"); ax[1].set_title("Responders (LDL down >= 15%)")
plt.tight_layout(); plt.show()

Q3 responders: treatment 52.3% vs placebo 17.1%; z=4.80, one-sided p=7.98e-07
   95% CI for the gap: [+22.0, +48.4] pts

All three agree. LDL fell about 29 mg/dL on treatment (paired, p tiny). That is about 18 mg/dL more than placebo (95% CI roughly 14 to 23), so it is the drug, not regression to the mean. And the responder rate is 52% vs 17% (z ≈ 4.8). The placebo arm's own ~10 mg/dL drop is exactly why a controlled comparison matters.

📋 STATISTICIAN’S REPORT

Conclusion: the treatment works, on all three measures

What we found. Patients on the treatment lowered their LDL by about 29 mg/dL over 12 weeks. The part we can credit to the drug, beyond what placebo patients dropped on their own, is about 18 mg/dL. And 52% of treated patients were responders (a 15%+ reduction) versus only 17% on placebo.

How confident are we? Extremely; every comparison is far beyond the threshold for chance, and we confirmed the arms started balanced. The 95% range for the drug's extra benefit is about 14 to 22 mg/dL, meaningful even at the low end.

Why the placebo arm mattered. Placebo patients also improved by ~10 mg/dL; comparing only before-versus-after on treatment would have over-credited the drug by that amount. The randomized, controlled comparison isolates the true effect.

Caveats. This is a 12-week surrogate marker (LDL), not long-term cardiovascular outcomes; safety and durability are separate questions.

---

Statistics, Data Science and AI: A Visual Handbook · © 2026 John Fisher

←Back to the bookChapter 86 · Case Study: A Clinical Trial