The Random Variables & Expectation chapter built random variables from scratch, an outcome, a value, a probability for each. But you rarely start from scratch. Most real counts follow one of a few standard patterns, and statisticians long ago cataloged them as named distributions: ready-made recipes, each with a formula, a mean, and a variance.
Recognizing that "this is a binomial" or "this is a Poisson" hands you the mean, the variance, and the exact probabilities for free, no derivation required. The hard part of a counting problem is almost never the algebra; it is spotting which recipe applies. This chapter is about building that recognition.
Distributions as Families
Each named distribution is a family: one formula, with knobs. Turn the knobs (the parameters) and you get a specific PMF with its own shape, center, and spread. A few have shapes worth knowing on sight.
Every one is summarized by two numbers from the Random Variables & Expectation chapter: its mean (where it centers) and its variance (how it spreads). The rest of this chapter is really just five recipes, each with its formula and those two summaries.
Bernoulli and Binomial
The Bernoulli distribution is the atom: a single trial that succeeds with probability p and fails with 1 − p. Stack n independent Bernoulli trials and count the successes, and you have the binomial distribution, the most important discrete distribution there is.
The formula reads naturally: C(n, k) counts the ways to arrange k successes among n trials (the combinatorics of the Counting & Combinatorics chapter), and p⁷(1−p)ⁿ⁻⁷ is the probability of any one such arrangement. For n = 10 and p = 0.3 the distribution centers on np = 3, with variance np(1 − p) = 2.1.
Waiting Games: Geometric, Negative Binomial, Hypergeometric
The binomial fixes the number of trials and counts successes. Flip the question, fix the number of successes and count the trials, and you get a different family.
The geometric distribution counts the trials up to the first success: PMF (1−p)k−1p, with mean 1/p. With p = 0.2 the most likely single outcome is success on the first try, yet the long right tail drags the average wait to 5 trials. Two close relatives complete the picture:
| Distribution | Counts | Key fact |
|---|---|---|
| Geometric | trials until the first success | mean 1/p; memoryless |
| Negative binomial | trials until the r-th success | generalizes the geometric (r = 1) |
| Hypergeometric | successes when drawing without replacement | trials are dependent, so not binomial |
The binomial assumes every trial has the same probability p, which holds only with replacement. Draw without replacing (cards from a deck, defective parts from a box) and each draw shifts the odds, so you need the hypergeometric. Drawing 10 parts from 50 with 5 defective gives P(exactly 1 defective) = 0.431. When the population is large relative to the sample, the difference fades and the binomial is a fine approximation.
The Poisson Distribution
The Poisson distribution counts events in a fixed window when they occur at a steady average rate λ: calls per hour, typos per page, mutations per genome. Its PMF is λ⁷e−λ/k!, and it has a beautiful signature: mean = variance = λ.
The Poisson is the limit of a binomial with many trials, each unlikely, holding np = λ fixed. That is exactly the structure of "rare events, many chances", which is why it models so much of the world. Here Binomial(1000, 0.003) and Poisson(3) are visually identical; their PMFs differ by less than 0.0004.
Choosing a Distribution
The skill is matching a problem to its distribution. Once matched, mean, variance, and exact probabilities all follow. Use this guide.
| If you are counting… | under these conditions | use |
|---|---|---|
| a single yes/no result | one trial, probability p | Bernoulli |
| successes in n tries | n independent trials, fixed p | Binomial |
| tries until the first success | independent trials, probability p | Geometric |
| tries until the r-th success | independent trials, probability p | Negative binomial |
| successes drawn without replacement | finite population, K successes | Hypergeometric |
| rare events in a fixed window | steady average rate λ | Poisson |
Discrete Distributions in Machine Learning & AI
These distributions are not academic. In machine learning, choosing a distribution for your data is choosing a model and a loss function, so the recipes in this chapter turn up all over modern AI.
| Distribution | In ML / AI it becomes | Concrete example |
|---|---|---|
| Bernoulli / Binomial | The likelihood behind binary classification | logistic regression; cross-entropy = −log Bernoulli likelihood |
| Categorical / Multinomial | A softmax output layer | the next-token distribution in a language model |
| Poisson | Count regression | predicting clicks, arrivals, or defect counts |
| Geometric / neg. binomial | Waiting and overdispersed counts | sessions until a user converts |
Every probabilistic model assumes a distribution for its output, and that choice silently fixes the loss function. Assume Bernoulli outputs and maximizing the likelihood becomes cross-entropy, the loss for classifiers and language models. Assume a Gaussian (the next chapter) and it becomes mean squared error. Assume Poisson and you get Poisson regression for counts. A neural network's softmax layer is literally a categorical distribution over classes, and a generative model's whole job is to learn a distribution it can sample from. The distributions in this chapter are the vocabulary in which models state what they believe about their data.
Build the distributions in Python
The companion notebook constructs the binomial, geometric, Poisson, and hypergeometric PMFs with SciPy, confirms each mean and variance by simulation, overlays Binomial(1000, 0.003) on Poisson(3) to watch them merge, compares drawing with and without replacement, and maximizes a Bernoulli likelihood, the same calculation that trains a logistic-regression classifier.
View opens the rendered notebook instantly (no setup). Open in Colab runs &
edits it live in your browser. To run locally, install numpy, scipy, and
matplotlib and launch jupyter notebook.
🎓 Key Takeaways
- ✓A named distribution is a family of PMFs sharing one formula, selected by parameters; knowing it gives you the mean and variance for free.
- ✓Binomial counts successes in n fixed trials: mean np, variance np(1 − p). Its atom is the Bernoulli trial.
- ✓Geometric counts trials to the first success (mean 1/p); the negative binomial waits for r successes; the hypergeometric handles sampling without replacement.
- ✓Poisson counts rare events at rate λ (mean = variance = λ) and is the large-n, small-p limit of the binomial.
- ✓In ML/AI, the assumed distribution sets the loss: Bernoulli → cross-entropy, Poisson → count regression, categorical → the softmax layer.
Practice Challenges
Five short challenges, beginner to intermediate. Try them on paper or with SciPy before checking the solutions.
Binomial
Roll a fair die 5 times. Find P(exactly 3 sixes) and the expected number of sixes.
Geometric
Each attempt succeeds with probability 0.25. Find the expected number of attempts to the first success and P(first success on attempt 3).
Poisson
A call center averages 4 calls per hour. Find P(exactly 2 calls in an hour) and P(no calls).
Hypergeometric
A box has 50 parts, 5 defective. You draw 10 without replacement. Find P(exactly 1 defective).
Maximum likelihood
A classifier is correct on 8 of 20 cases. What is the maximum-likelihood estimate of its success probability p?
A fully-worked solutions notebook walks through all five challenges, each verified in code. Try them yourself first, then compare.
Quiz: Test Yourself
Eight quick questions on discrete distributions. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.