Contents/ Part VI · Probability/ Chapter 34

Conditional Probability & Independence

How does knowing one fact change the probability of another? Conditional probability is the answer, and it is the idea behind tree diagrams, the notorious base-rate trap, and every classifier in machine learning.

⏱️ ~15 min read
🐍 Notebook included
📊 Chapter 34

Most real probabilities are conditional. We rarely ask "what is the chance of rain?" in a vacuum; we ask given the clouds, given the season, given the forecast. Conditional probability makes "given" precise, and it turns out to be the hinge between basic probability and machine learning.

|
P(A | B) is the probability of A given that B has happened, defined as P(A | B) = P(A and B) / P(B). Two events are independent when conditioning changes nothing: P(A | B) = P(A).
🔎
Conditioning shrinks the world

The whole idea: once you know B happened, the only outcomes that remain possible are the ones inside B. So you recompute A's probability using B as the new, smaller sample space. That single shift, from the full space to "the world where B is true", explains tree diagrams, the base-rate trap, and how a spam filter reasons.

1

Conditional Probability: P(A | B)

To find P(A | B), restrict attention to the outcomes where B is true, then ask what fraction of those also have A. The denominator is no longer the whole sample space, it is just B.

Given B, the world shrinks to B sample space S A B given B A and B B only B is now everything; A's chance is the shaded share

In the notebook, among 1,000 visitors only 11% buy, but among the 400 who saw an ad, 20% buy: P(buy | saw ad) = 80 / 400 = 0.20. The denominator dropped from 1,000 to 400 because we conditioned on the ad. That the conditional probability differs from the overall one is precisely what makes the events dependent.

2

The Multiplication Rule & Tree Diagrams

Rearranging the definition gives the general multiplication rule, P(A and B) = P(B) × P(A | B). A probability tree draws it: the chance of a path is the product of its branch probabilities, and later branches are conditional on earlier ones.

Two cards, no replacement: the second branch is conditional on the first deck P(ace) = 4/5248/52 P(ace | ace) = 3/5148/51 ace other ace both aces P(ace and ace) = 4/52 × 3/51 ≈ 0.0045 (about 1 in 221)

Removing the first ace changes the deck, so the second probability is the conditional 3/51, not 4/52. If you drew with replacement the draws would be independent and you would multiply 4/52 by 4/52 instead. The tree makes the dependence visible: the second fork depends on which way the first went.

3

Independence, Through Conditioning

Rules of Probability tested independence with P(A and B) = P(A) × P(B). Conditional probability gives the more intuitive version of the same idea.

🟰
Independent means "the condition tells you nothing"

A and B are independent exactly when P(A | B) = P(A): knowing B happened does not change A's probability at all. Two dice are independent, P(die 2 is a six) stays 1/6 whether or not die 1 is even. The card draw is dependent: P(second is an ace) falls from 4/52 to 3/51 once you learn the first was an ace. Same test as before, stated as "does conditioning move the needle?"

This is also the cleanest way to spot dependence in data: compare P(A | B) to P(A). If they differ, B carries information about A, which is exactly the signal a model wants to learn.

4

The Base-Rate Trap

Now the chapter's headline: a test can be 99% accurate and still be wrong half the time it says "positive". The reason is conditional probability colliding with a rare event.

10,000 people · 1% have the disease · test is 99% accurate 10,000 people 1%99% 100 sick 9,900 healthy 99 test +true positive 1 test - 99 test +false positive 9,801 test - P(disease | positive) = 99 / (99 + 99) = 50%

The 99 true positives are exactly matched by 99 false positives, because a 1% error rate applied to the huge healthy majority produces just as many. So a positive result means only a 50% chance of disease. This is the base-rate fallacy: a 1978 study found most Harvard physicians answered around 95%, ignoring how rare the disease was. The fix is to count natural frequencies, true against false positives, which the next chapter turns into Bayes' Theorem.

5

Conditional Probability in Machine Learning & AI

Conditional probability is not a side topic for machine learning, it is the main event. Most models exist to estimate one conditional probability or another.

A classifier estimates P(class | features): one word can move it a lot P(spam) 0.30 P(spam | "free") 0.72 seeing one word updates the conditional probability of spam; a model combines many such words.
Conditional ideaIn ML / AI it appears asConcrete example
P(class | features)What every classifier estimatesspam filter: P(spam | the words in the email)
Conditional independenceThe "naive" in naive Bayesfeatures assumed independent given the class
P(next | previous)Language models and autoregressionan LLM predicts P(next token | the context so far)
Conditional dependenceBayesian networks, graphical modelsencode which variables depend on which
Base rate P(class)Class imbalance and calibrationon a rare class, accuracy hides the screening trap
🤖
Why this matters for AI research

A large language model is, at heart, an enormous estimator of P(next token | everything so far), generating text by sampling from that conditional distribution one token at a time. Naive Bayes is named for its conditional-independence assumption, that features are independent given the class, which is usually false yet works surprisingly well. And the base-rate trap returns as the precision problem on imbalanced data: a fraud detector that is "99% accurate" can still flag mostly false alarms when fraud is rare. Reading P(A | B) correctly, base rate and all, is a core research skill.

🐍

Condition it in Python

The companion notebook reads P(A | B) off a contingency table, draws two cards without replacement (and checks the multiplication rule by simulation), tests independence by comparing P(A | B) with P(A), works the full base-rate screening example with a prevalence curve, and computes P(spam | "free") the way naive Bayes does.

📓 View Notebook (code & outputs) ▶ Open in Colab ⬇ View / Download on GitHub

View opens the rendered notebook instantly (no setup). Open in Colab runs & edits it live in your browser. To run locally, install numpy and matplotlib and launch jupyter notebook.

🎓 Key Takeaways

  • P(A | B) = P(A and B) / P(B): conditioning restricts the world to B and recomputes A inside it.
  • Multiplication rule: P(A and B) = P(B) × P(A | B); trees multiply along conditional branches.
  • Independence means P(A | B) = P(A): the condition changes nothing.
  • Base-rate trap: a 99%-accurate test for a 1% disease gives P(disease | positive) = 50%, not 99%.
  • In ML/AI: classifiers estimate P(class | features); LLMs estimate P(next | context); naive Bayes assumes conditional independence.
6

Practice Challenges

Five short challenges, beginner to intermediate. Try them on paper or in Python before checking the solutions.

1

From a table

Of 200 students, 120 study Python, 90 study Python and statistics, 130 study statistics. Find P(statistics | Python). Is it higher or lower than P(statistics)?

Hint: condition on Python, so divide by 120.
2

Without replacement

A bag has 5 red and 3 blue marbles. Draw 2 without replacement. Find P(both red) and verify by simulation.

Hint: (5/8) × (4/7).
3

Independent?

Draw one card. Let A = {King}, B = {Heart}. Is A independent of B? Check whether P(A | B) = P(A).

Hint: P(King) = 4/52, P(King | Heart) = 1/13.
4

Base rate

A disease has 2% prevalence; a test has 95% sensitivity and 90% specificity. In a town of 10,000, find P(disease | positive).

Hint: count true positives vs false positives, then take the ratio.
5

P(class | feature)

Of 500 reviews, 200 are negative. The word "refund" appears in 120 negative and 30 positive reviews. Find P(negative | "refund").

Hint: of the reviews with "refund", what fraction are negative?
Check your work

A fully-worked solutions notebook walks through all five challenges, each verified in code. Try them yourself first, then compare.

📓 View Solutions ▶ Open Solutions in Colab ⬇ View / Download on GitHub
7

Quiz: Test Yourself

Eight quick questions on conditional probability and independence. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.