Reproducibility: Seeds, Environments, Data
import numpy as np, pandas as pd, sys, platform, hashlib
import importlib.metadata as meta
DEMO 1 · The bug: randomness with no seed¶
Any analysis that samples, splits, shuffles, or initializes at random gives a different answer each run unless you fix the seed. Watch an unseeded 'model score' wobble, then pin it and watch it lock.
def train_score(seed=None):
rng = np.random.default_rng(seed)
return round(0.80 + rng.normal(0, 0.02), 4) # stand-in for a real model's test score
print("no seed: ", train_score(), train_score(), " <- different every run")
print("seed=42: ", train_score(42), train_score(42), " <- identical, reproducible")
no seed: 0.7953 0.7866 <- different every run seed=42: 0.8061 0.8061 <- identical, reproducible
DEMO 2 · Capture the environment¶
The same code can give different answers under different library versions. So record the environment: the Python version and the versions of every package that matters. This is the report you attach to any result.
def environment_report(packages=("numpy", "pandas")):
report = {"python": sys.version.split()[0], "platform": platform.system()}
for p in packages:
try: report[p] = meta.version(p)
except meta.PackageNotFoundError: report[p] = "not installed"
return report
env = environment_report()
for k, v in env.items(): print(f"{k:10} {v}")
python 3.14.2 platform Darwin numpy 2.4.4 pandas 3.0.3
DEMO 3 · Pin dependencies with requirements.txt¶
To let someone rebuild your environment exactly, list every package at an exact version. A requirements.txt with == pins is the standard; pip install -r requirements.txt then reproduces it. The looseness of >= is what breaks reproducibility.
reqs = "\n".join(f"{p}=={v}" for p, v in env.items() if p not in ("python", "platform"))
print("requirements.txt")
print("-" * 20)
print(reqs)
print("\nTip: a virtual environment (python -m venv .venv) keeps these pins isolated per project.")
requirements.txt -------------------- numpy==2.4.4 pandas==3.0.3 Tip: a virtual environment (python -m venv .venv) keeps these pins isolated per project.
DEMO 4 · Version the data with a hash¶
Code and environment are not enough; the data must be the same too. A hash is a short fingerprint of a file's exact contents. Store it, and you can prove later whether the data changed, even by a single digit.
def data_hash(df):
return hashlib.sha256(df.to_csv(index=False).encode()).hexdigest()[:12]
df = pd.DataFrame({"x": [1, 2, 3], "y": [10, 20, 30]})
h1 = data_hash(df)
print("hash of original data:", h1)
df2 = df.copy(); df2.loc[0, "y"] = 11 # change ONE value
print("hash after a 1-cell edit:", data_hash(df2))
print("same data reproduces the hash:", data_hash(df.copy()) == h1)
hash of original data: fcea3980d2b6 hash after a 1-cell edit: a4a1fe69a6db same data reproduces the hash: True
DEMO 5 · Stamp every result with its provenance¶
Put it together: a small function that tags a result with everything needed to reproduce it, the seed, the data hash, and the environment. Attach this to your outputs and 'it works on my machine' stops being a mystery.
def run_experiment(df, seed):
return {"score": train_score(seed), "seed": seed,
"data_hash": data_hash(df), "env": environment_report()}
result = run_experiment(df, seed=42)
import json; print(json.dumps(result, indent=2))
print("\nRerun with the same seed, data, and environment and the score is guaranteed to match.")
{
"score": 0.8061,
"seed": 42,
"data_hash": "fcea3980d2b6",
"env": {
"python": "3.14.2",
"platform": "Darwin",
"numpy": "2.4.4",
"pandas": "3.0.3"
}
}
Rerun with the same seed, data, and environment and the score is guaranteed to match.
Wrap-up¶
Three of the four pillars: a fixed seed makes randomness repeatable, a pinned environment (requirements.txt in a virtualenv) makes the libraries repeatable, and a data hash proves the inputs did not change. Stamp your results with all three and anyone can reproduce them. The fourth pillar, versioning the code itself, is what git does, and that is next.