Why We Sample argued that representativeness is everything. Probability sampling is how you guarantee it: give every unit a known, non-zero chance of selection, and chance, not convenience or judgment, decides who is in.
The notebook runs all four on the same 100,000-person population (mean income $55,862) at the same sample size n = 500, so their precision is directly comparable. Each is unbiased; what differs is the spread.
Simple Random Sampling (SRS)
The purest method: every possible sample of size n is equally likely, like drawing names from a hat. SRS is unbiased and is the benchmark every other method is measured against.
In the notebook, 2,000 SRS samples of 500 people center exactly on the true mean, with a standard error of $789. SRS needs a complete list of the population (a sampling frame) and can be costly to reach scattered units, the practical motivation for the methods that follow.
Stratified Sampling
Divide the population into strata, internally similar groups (regions, age bands, customer tiers), and sample each one separately, usually in proportion to its size. Because each stratum is homogeneous, the between-group differences stop adding noise, and precision jumps.
The numbers are striking: stratified sampling cuts the standard error from $789 to $400, a 74% reduction in variance, simply by guaranteeing each region appears in the right proportion. Stratified sampling never does worse than SRS, and the more the strata differ, the larger the gain. The cost is that you must know the strata in advance.
Cluster Sampling
When reaching scattered individuals is expensive, cluster sampling picks a few whole groups, city blocks, schools, clinics, and surveys everyone inside them. It is cheap to run, but it pays in precision.
With the same 500 units, cluster sampling's standard error balloons to $7,847, roughly ten times SRS. The reason: people in one cluster resemble each other, so 100 neighbors carry far less information than 100 independent draws. You accept cluster sampling when the cost savings outweigh the precision loss, and you often recover some precision by sampling more clusters with fewer units each.
Systematic Sampling & the Periodicity Trap
Systematic sampling takes every k-th unit after a random start (k = N/n). It is simple to administer, every 200th customer, and on a randomly ordered list it behaves just like SRS. The danger is a hidden cycle.
On a shuffled list the notebook's systematic estimate has a standard error of $794, essentially identical to SRS. But on a list that cycles with period k, every selected unit shares the same value and the estimate collapses to $30,000 against a true $60,000. The fix is simple: shuffle the list, or check it for periodic structure (seasonal data, alternating records) before stepping through it.
Probability Sampling in Machine Learning & AI
These methods are everyday tools in machine learning, usually under different names. Splitting data, balancing classes, and cross-validation are all probability sampling in disguise.
| Method (this chapter) | In ML / AI it becomes | Why it helps |
|---|---|---|
| Simple random sampling | Random train / test split | an unbiased estimate of generalization |
| Stratified sampling | Stratified split & stratified k-fold | keeps class proportions in every fold, lower-variance metrics |
| Proportional allocation | Class-balanced batches | rare classes still appear each batch |
| Cluster sampling | Group-aware splits (GroupKFold) | keeps related rows (same user) together to avoid leakage |
| Systematic sampling | Strided sampling of streams / logs | cheap subsampling, watch for periodicity |
The single most common split, train_test_split, is SRS, and its stratify= option is
stratified sampling, indispensable when a class is rare so every fold keeps the same balance. Reducing
evaluation variance with stratified k-fold is exactly the stratification win from section 2.
The cluster idea returns as group-aware splitting: if many rows belong to the
same user, splitting them randomly leaks information across train and test, so you sample whole groups instead.
And the periodicity trap reappears whenever you subsample time-ordered logs with a fixed
stride. The vocabulary changes; the principles do not.
Run all four methods in Python
The companion notebook builds a 100,000-person stratified population, then runs simple random, stratified, cluster, and systematic sampling 2,000 times each, comparing their standard errors and reproducing the periodicity trap, all at the same sample size.
View opens the rendered notebook instantly (no setup). Open in Colab runs &
edits it live in your browser. To run locally, install numpy, pandas, and
matplotlib and launch jupyter notebook.
🎓 Key Takeaways
- ✓Probability sampling gives every unit a known, non-zero chance of selection, making the sample representative and the error calculable.
- ✓SRS is the unbiased benchmark (SE $789 here); it needs a full sampling frame.
- ✓Stratified sampling cut the SE to $400 (74% less variance) by representing each group in proportion; it never beats worse than SRS.
- ✓Cluster sampling is cheap but noisy (SE $7,847) because units within a cluster are alike.
- ✓Systematic sampling matches SRS on a shuffled list but fails on a periodic one; in ML these are train/test splits, stratified k-fold, and group-aware splits.
Practice Challenges
Five short challenges, beginner to intermediate. Try them with NumPy before checking the solutions.
SRS is unbiased
Draw 3,000 simple random samples of size 200 from a two-stratum population and confirm the average estimate equals the true mean.
Stratified beats SRS
Draw stratified samples (proportional allocation, n = 200) and compare the standard error to SRS.
The cluster penalty
Split the sorted population into homogeneous clusters of 100, sample two clusters, and show the SE exceeds SRS at the same n.
The periodicity trap
Build a list whose values repeat with period k and show systematic sampling (every k-th) returns a biased estimate.
Proportional allocation
Verify that proportional allocation samples each stratum in proportion to its share of the population.
A fully-worked solutions notebook walks through all five challenges, each verified in code. Try them yourself first, then compare.
Quiz: Test Yourself
Eight quick questions on probability sampling. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.
These methods all rely on known selection probabilities. The Non-Probability Sampling Methods chapter turns to convenience, voluntary, quota, and snowball sampling, which are cheaper and common, but sacrifice the very property that makes error calculable.