The geometric waits for the first success. But what if you need a quota, ten passing tests before you ship? Counting the trials (or the failures) on the way to r successes is the negative binomial, the geometric scaled up to a goal.
software_reliability_testing.csv records 1,000 suites, each run until
target_successes_r = 10 cases pass, logging failures_encountered and
total_tests_run. We model the failures as negative binomial.
Trials to a Quota
Since total tests = 10 passes + failures, the per-test pass probability is p = r / mean(total) = 0.79. The negative binomial built on that p reproduces the observed failures, averaging 2.6 failures before the tenth pass.
The PMF C(k + 9, k) p10(1−p)k drops onto the data. It is the geometric (one success) generalized to a quota of ten, and its variance (3.34) exceeds its mean (2.64), a hallmark of overdispersion.
Deployment Readiness
Total tests = 10 passes + the failures along the way, so on average the suite needs about 13 runs (r/p) to bank 10 passes. The CDF then converts a test budget into a clear-probability, the number a release manager actually wants.
| Test budget | failures allowed | P(suite clears) |
|---|---|---|
| 12 runs | ≤ 2 | 0.527 |
| 14 runs | ≤ 4 | 0.850 |
| 16 runs | ≤ 6 | 0.967 |
| 18 runs | ≤ 8 | 0.994 |
A 14-run budget clears the suite about 85% of the time; 16 runs reach 97% and 18 runs about 99%. The negative binomial turns "is it ready to ship?" from a gut call into a budgeted probability, the foundation of a sane release gate.
A Sum of Geometrics
The structure explains the shape. A negative binomial for r successes is the sum of r independent geometric waits, one for each pass. Simulating ten stacked geometric waits reproduces the distribution exactly.
Each pass is quick on average, but the occasional run of failures compounds across the ten waits, giving the negative binomial a longer right tail than a Poisson of the same mean (variance 3.34 vs mean 2.64). That extra spread, overdispersion, is exactly why the negative binomial is the standard model for counts that are more variable than Poisson allows, from defect clustering to insurance claims.
Run the reliability audit
The companion notebook fits the per-test pass rate, overlays the negative binomial on the observed failures, validates the fit with a chi-square goodness-of-fit test, converts test budgets into clear-probabilities with the CDF, and rebuilds the distribution as ten stacked geometric waits, confirming the mean and exposing the overdispersed tail.
View opens the rendered notebook instantly (no setup). Open in Colab runs &
edits it live in your browser. To run locally, install numpy, pandas, scipy,
and matplotlib and launch jupyter notebook.
🎓 Key Takeaways
- ✓Trials to a quota of r successes are negative binomial: PMF C(k+r−1, k) pr(1−p)k.
- ✓Fit p from the mean: p = r/mean(total) = 0.79; mean failures r(1−p)/p = 2.6, mean total r/p = 13.
- ✓The CDF gives readiness: a 14-run budget clears the suite 85% of the time, 18 runs 99%.
- ✓It is a sum of r geometrics, which is why it generalizes the geometric and has a heavy right tail.
- ✓Variance > mean (overdispersion), the reason it models counts too variable for a Poisson.
Quiz: Test Yourself
Eight quick questions on the negative binomial case study. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.