The Probability Fundamentals chapter gave us single probabilities. Real questions almost always combine events: the chance of this or that, of this and that, of at least one. A small set of rules, all flowing from Kolmogorov's axioms, handles every case.
"Or" means at least one of the events happens, that is the addition rule. "And" means both happen together, that is the multiplication rule. Almost every probability calculation begins by spotting which keyword the question is really asking, then applying the matching rule and watching for the two classic traps (double-counting, and confusing exclusivity with independence).
The Addition Rule: P(A or B)
To find the probability that A or B happens, you add their probabilities, but if they can both happen you must subtract the overlap, or you count it twice.
For the card example in the notebook, P(heart or face card) = 13/52 + 12/52 − 3/52 = 22/52. The three cards that are both a heart and a face card (the jack, queen, and king of hearts) would otherwise be double-counted.
The Multiplication Rule: P(A and B)
To find the probability that A and B both happen, you multiply. A probability tree makes it visual: the chance of a path is the product of the probabilities along its branches.
The full multiplication rule is P(A and B) = P(A) × P(B | A), where P(B | A) is the probability of B given A already happened (the Conditional Probability & Independence chapter develops this). When A and B are independent, A tells you nothing about B, so P(B | A) = P(B) and the rule simplifies to the plain product P(A) × P(B). Two dice both even: 1/2 × 1/2 = 1/4.
Mutually Exclusive vs Independent
These two terms sound similar and get swapped all the time, but they describe opposite kinds of relationship. Getting them straight is one of the most valuable things in this chapter.
| Mutually exclusive | Independent | |
|---|---|---|
| Can both occur? | No, never | Yes |
| P(A and B) | = 0 | = P(A) × P(B) |
| Relationship | Strongly dependent (one rules the other out) | No relationship |
| Example | A die showing even vs odd | Two separate coin flips |
Mutually exclusive events are actually dependent: if A happens, B becomes impossible, so A changes B's probability dramatically. Independence is the opposite situation, where A changes nothing about B. The only safe move is to test: events are independent precisely when P(A and B) = P(A) × P(B). Never assume it from the wording.
Putting the Rules Together
Most problems are solved by picking the right rule, sometimes two in sequence. A quick decision guide:
The "at least one" pattern is worth memorizing because it shows up everywhere. P(at least one six in four rolls) = 1 − (5/6)⁴ ≈ 0.52: the chance of no sixes is an AND of four independent misses, and the complement turns it into "at least one". It crosses 50% at the fourth roll.
Probability Rules in Machine Learning & AI
The four moves you just learned, OR, AND, independence, and "at least one", are the grammar of real probability models. Two of them sit right at the heart of how a spam filter and a language model work.
| Rule (this chapter) | In ML / AI it becomes | Concrete example |
|---|---|---|
| Multiplication (AND) | How naive Bayes scores a class | multiply P(word | spam) across all the words |
| Independence assumption | The "naive" in naive Bayes | words treated as independent given the class |
| Chain rule (repeated AND) | How a language model factors a sentence | P(sentence) = product of P(word | previous words) |
| Addition (OR) | Total probability over disjoint classes | sum P across mutually exclusive labels |
| Complement ("at least one") | Ensembles and redundancy | 1 − P(every detector misses) |
Naive Bayes is the multiplication rule plus a deliberately simple independence assumption that is usually false yet works remarkably well. The chain rule, just the multiplication rule applied over and over, is exactly how a language model assigns a probability to a sentence, multiplying one conditional factor per token. And "at least one" logic explains why stacking independent checks (ensembles, redundant safeguards) sharply raises the odds of catching a rare event. The same four moves scale all the way up to the algorithms later in the book.
See the rules in action
The companion notebook works the addition rule on a card deck (with a Venn picture of the overlap), checks the mutually exclusive special case on a die, multiplies independent events across two dice, runs the independence test that separates "mutually exclusive" from "independent", and builds an "at least one" curve from the complement, each verified by simulation.
View opens the rendered notebook instantly (no setup). Open in Colab runs &
edits it live in your browser. To run locally, install numpy and matplotlib and launch
jupyter notebook.
🎓 Key Takeaways
- ✓Addition (OR): P(A or B) = P(A) + P(B) − P(A and B); subtract the overlap so it is not double-counted.
- ✓Multiplication (AND): P(A and B) = P(A) × P(B | A); for independent events this is just P(A) × P(B).
- ✓Mutually exclusive ≠ independent: exclusive means P(A and B) = 0 (and therefore dependent); independent means P(A and B) = P(A) × P(B).
- ✓Test independence, never assume it: check whether P(A and B) equals P(A) × P(B).
- ✓"At least one" = 1 − P(none): combine the multiplication rule with the complement.
Practice Challenges
Five short challenges, beginner to intermediate. Try them on paper or in Python before checking the solutions.
Addition with overlap
Draw one card. Find P(red or king). Identify the overlap, apply the addition rule, and verify by simulation.
Mutually exclusive
Roll one die. Find P(roll a 2 or a 5). Are these mutually exclusive? What happens to the overlap term?
Multiplication (independent)
Flip a coin and roll a die. Find P(heads and a six). Are they independent? Verify by simulation.
Test independence
Given P(A)=0.5, P(B)=0.4: in case (i) P(A and B)=0.2, in case (ii) P(A and B)=0. For each, is the pair independent, mutually exclusive, or neither?
At least one
A shooter makes 80% of free throws. Assuming independent shots, find P(at least one miss in 5 attempts) using the complement.
A fully-worked solutions notebook walks through all five challenges in the same visual style. Try them yourself first, then compare.
Quiz: Test Yourself
Eight quick questions on the rules of probability. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.