Bayesian Inference in Practice

Every estimate you have made so far, a mean, a proportion, a regression slope, has been a single number with a confidence interval bolted on. The Bayesian view keeps the whole distribution of plausible values instead. You start with a prior, what you believed before seeing the data; you weigh it by the likelihood, how well each value explains the data; and you end with a posterior, your updated belief. Everything else, best guesses and uncertainty alike, is just a question you ask of that posterior.

Start with the simplest real question: what is the true conversion rate of a landing page? Call it θ. Before collecting data you hold a prior, a distribution over the values θ might take. Then 40 visitors arrive and 8 convert. The likelihood scores every candidate rate by how well it explains “8 out of 40.” Multiply the two, normalize, and you have the posterior.

Because a Beta prior and a Binomial likelihood are a conjugate pair, the posterior is again a Beta, no integration required: a Beta(a, b) prior plus c successes in n trials becomes Beta(a + c, b + n − c). Here a weak Beta(2, 2) prior and 8 of 40 give Beta(10, 34). That single object answers every question: its mean is 0.227, and the middle 95 percent, the credible interval, runs from 0.118 to 0.360. Want the chance the rate beats 30 percent? It is the posterior area above 0.30, about 13 percent. No sampling distribution, no null hypothesis, just areas under one curve.

People often want “the number.” The posterior offers three, and the difference between them is the whole lesson. Two are single points on the curve; the third is the curve itself.

• ●MLE (maximum likelihood): the value that best fits the data alone, ignoring the prior. Here 8/40 = 0.200, the peak of the likelihood. It is what classical statistics reports.
• ●MAP (maximum a posteriori): the peak of the posterior, the single most probable value once the prior is folded in. Here 0.214. It is a regularized MLE, the prior nudges it toward the middle.
• ●The full posterior: the whole Beta(10, 34) curve, mean 0.227. This is the Bayesian answer, because only the full distribution carries the uncertainty that the two points throw away.

Conjugate pairs like Beta-Binomial give the posterior in closed form, but they are the lucky exceptions. The moment a model has several parameters, hierarchical structure, or an awkward likelihood, the normalizing integral is impossible to do by hand. The Bayesian workhorse is to sample from the posterior instead of solving for it, and a pile of samples answers every question a formula could.

The simplest sampler, Metropolis-Hastings, is almost embarrassingly simple: propose a small random step, and accept it if the posterior there is higher, or with a matching probability if it is lower. Do that tens of thousands of times and the chain visits each region of parameter space in proportion to its posterior probability, so the collected draws are a sample from the posterior. Modern tools use a far smarter version, Hamiltonian Monte Carlo (HMC) and its self-tuning cousin NUTS, which use the gradient of the posterior to take long, efficient strides instead of a random shuffle.

In practice you never write the sampler yourself. A probabilistic programming language lets you declare the model and hands the inference to a battle-tested engine:

import pymc as pm
with pm.Model() as model:
    theta = pm.Beta("theta", alpha=2, beta=2)          # prior
    pm.Binomial("y", n=40, p=theta, observed=8)        # likelihood
    idata = pm.sample(2000, tune=1000, chains=4)       # NUTS does the rest

Because you are trusting a sampler rather than a formula, you must check that it converged. Two diagnostics do most of the work: the trace plot should look like a fuzzy caterpillar, the chains overlapping with no drift or stuck stretches; and R-hat compares variation within each chain to variation between chains, with values at 1.00 (below about 1.01) signaling agreement. A healthy run here recovers the same posterior the conjugate formula gave, mean near 0.227, which is the whole point: MCMC is a general-purpose route to the answer you could only sometimes get exactly.

The Bayesian payoff is clearest in an A/B test, where the question is not “is there a significant difference” but the far more useful “how likely is B better, and by how much is it worth?” The companion notebook works a full campaign end to end with conjugate posteriors and Monte Carlo.

Contrast the two write-ups. The frequentist test returns a two-proportion z = 2.70, p = 0.007, “reject the null of no difference,” true but thin. The Bayesian analysis hands a stakeholder a sentence they can act on: “There is a 99.6 percent chance B is better, and about a 92 percent chance the lift is at least 2 points.” Finally, a posterior predictive check closes the loop: simulate new campaigns from the fitted posteriors and confirm the conversions they generate look like the data you actually saw. If the real data sits in the fat part of that simulated distribution, the model is telling a story consistent with reality.

Bayesian thinking is not a statistical backwater; it is woven through modern machine learning, usually wherever a model needs to say not just what it predicts but how sure it is. The prior-times-likelihood engine reappears under many names.

View opens the rendered notebook instantly. Open in Colab runs it live (PyMC is preinstalled there). To run locally, install numpy, scipy, pandas, matplotlib, and openpyxl; the PyMC cell is optional and skips itself if the library is absent.

Five exercises on the conversion and A/B examples. Full solutions are in the companion solutions notebook.

Eight questions on priors, posteriors, MLE/MAP, MCMC, and credible intervals. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved.