Most real probabilities are conditional. We rarely ask "what is the chance of rain?" in a vacuum; we ask given the clouds, given the season, given the forecast. Conditional probability makes "given" precise, and it turns out to be the hinge between basic probability and machine learning.
The whole idea: once you know B happened, the only outcomes that remain possible are the ones inside B. So you recompute A's probability using B as the new, smaller sample space. That single shift, from the full space to "the world where B is true", explains tree diagrams, the base-rate trap, and how a spam filter reasons.
Conditional Probability: P(A | B)
To find P(A | B), restrict attention to the outcomes where B is true, then ask what fraction of those also have A. The denominator is no longer the whole sample space, it is just B.
In the notebook, among 1,000 visitors only 11% buy, but among the 400 who saw an ad, 20% buy: P(buy | saw ad) = 80 / 400 = 0.20. The denominator dropped from 1,000 to 400 because we conditioned on the ad. That the conditional probability differs from the overall one is precisely what makes the events dependent.
The Multiplication Rule & Tree Diagrams
Rearranging the definition gives the general multiplication rule, P(A and B) = P(B) × P(A | B). A probability tree draws it: the chance of a path is the product of its branch probabilities, and later branches are conditional on earlier ones.
Removing the first ace changes the deck, so the second probability is the conditional 3/51, not 4/52. If you drew with replacement the draws would be independent and you would multiply 4/52 by 4/52 instead. The tree makes the dependence visible: the second fork depends on which way the first went.
Independence, Through Conditioning
Rules of Probability tested independence with P(A and B) = P(A) × P(B). Conditional probability gives the more intuitive version of the same idea.
A and B are independent exactly when P(A | B) = P(A): knowing B happened does not change A's probability at all. Two dice are independent, P(die 2 is a six) stays 1/6 whether or not die 1 is even. The card draw is dependent: P(second is an ace) falls from 4/52 to 3/51 once you learn the first was an ace. Same test as before, stated as "does conditioning move the needle?"
This is also the cleanest way to spot dependence in data: compare P(A | B) to P(A). If they differ, B carries information about A, which is exactly the signal a model wants to learn.
The Base-Rate Trap
Now the chapter's headline: a test can be 99% accurate and still be wrong half the time it says "positive". The reason is conditional probability colliding with a rare event.
The 99 true positives are exactly matched by 99 false positives, because a 1% error rate applied to the huge healthy majority produces just as many. So a positive result means only a 50% chance of disease. This is the base-rate fallacy: a 1978 study found most Harvard physicians answered around 95%, ignoring how rare the disease was. The fix is to count natural frequencies, true against false positives, which the next chapter turns into Bayes' Theorem.
Conditional Probability in Machine Learning & AI
Conditional probability is not a side topic for machine learning, it is the main event. Most models exist to estimate one conditional probability or another.
| Conditional idea | In ML / AI it appears as | Concrete example |
|---|---|---|
| P(class | features) | What every classifier estimates | spam filter: P(spam | the words in the email) |
| Conditional independence | The "naive" in naive Bayes | features assumed independent given the class |
| P(next | previous) | Language models and autoregression | an LLM predicts P(next token | the context so far) |
| Conditional dependence | Bayesian networks, graphical models | encode which variables depend on which |
| Base rate P(class) | Class imbalance and calibration | on a rare class, accuracy hides the screening trap |
A large language model is, at heart, an enormous estimator of P(next token | everything so far), generating text by sampling from that conditional distribution one token at a time. Naive Bayes is named for its conditional-independence assumption, that features are independent given the class, which is usually false yet works surprisingly well. And the base-rate trap returns as the precision problem on imbalanced data: a fraud detector that is "99% accurate" can still flag mostly false alarms when fraud is rare. Reading P(A | B) correctly, base rate and all, is a core research skill.
Condition it in Python
The companion notebook reads P(A | B) off a contingency table, draws two cards without replacement (and checks the multiplication rule by simulation), tests independence by comparing P(A | B) with P(A), works the full base-rate screening example with a prevalence curve, and computes P(spam | "free") the way naive Bayes does.
View opens the rendered notebook instantly (no setup). Open in Colab runs &
edits it live in your browser. To run locally, install numpy and matplotlib and launch
jupyter notebook.
🎓 Key Takeaways
- ✓P(A | B) = P(A and B) / P(B): conditioning restricts the world to B and recomputes A inside it.
- ✓Multiplication rule: P(A and B) = P(B) × P(A | B); trees multiply along conditional branches.
- ✓Independence means P(A | B) = P(A): the condition changes nothing.
- ✓Base-rate trap: a 99%-accurate test for a 1% disease gives P(disease | positive) = 50%, not 99%.
- ✓In ML/AI: classifiers estimate P(class | features); LLMs estimate P(next | context); naive Bayes assumes conditional independence.
Practice Challenges
Five short challenges, beginner to intermediate. Try them on paper or in Python before checking the solutions.
From a table
Of 200 students, 120 study Python, 90 study Python and statistics, 130 study statistics. Find P(statistics | Python). Is it higher or lower than P(statistics)?
Without replacement
A bag has 5 red and 3 blue marbles. Draw 2 without replacement. Find P(both red) and verify by simulation.
Independent?
Draw one card. Let A = {King}, B = {Heart}. Is A independent of B? Check whether P(A | B) = P(A).
Base rate
A disease has 2% prevalence; a test has 95% sensitivity and 90% specificity. In a town of 10,000, find P(disease | positive).
P(class | feature)
Of 500 reviews, 200 are negative. The word "refund" appears in 120 negative and 30 positive reviews. Find P(negative | "refund").
A fully-worked solutions notebook walks through all five challenges, each verified in code. Try them yourself first, then compare.
Quiz: Test Yourself
Eight quick questions on conditional probability and independence. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.