4.13 Bayes' theorem

4.13 Bayes' theorem

IB Maths: Analysis and Approaches HL: Revision notes

Studying this course? Keep your progress in one place.

Key facts

  • Bayes' theorem reverses a conditional probability: from P(A∣B)P(A\mid B) to P(B∣A)P(B\mid A).
  • Numerator: the one route through the cause of interest. Denominator: P(A)P(A), the sum of all routes.
  • Up to three events in a partition: B1B_1, B2B_2, B3B_3.
  • Use the law of total probability to find unknown probabilities first.
  • Interpret the result: rare causes give low posteriors even with accurate tests.

What Bayes' theorem solves

It turns P(evidence∣cause)P(\text{evidence}\mid\text{cause}) round into P(cause∣evidence)P(\text{cause}\mid\text{evidence}).

Often we know how likely an observation is given a cause, such as P(positive∣condition)P(\text{positive}\mid\text{condition}), but we want the reverse: P(condition∣positive)P(\text{condition}\mid\text{positive}). Bayes' theorem turns one conditional probability round into the other.

The starting probability of the cause, before the evidence, is the prior probability. The updated probability after the evidence is the posterior probability.

  1. 1

    Prior

    the probability of the cause before any evidence

  2. 2

    Evidence

    an observation, such as a positive test

  3. 3

    Posterior

    the updated probability of the cause given the evidence

Updating a probability

Which is the posterior in 'a person tests positive; how likely is the condition?'

Two events

The route through BB that produces AA, divided by the total probability of AA.

From the formula booklet:

P(B∣A)=P(B)P(A∣B)P(B)P(A∣B)+P(B′)P(A∣B′).P(B\mid A)=\frac{P(B)P(A\mid B)}{P(B)P(A\mid B)+P(B')P(A\mid B')}.

The numerator is the route through BB that produces AA. The denominator is P(A)P(A), found by the law of total probability: add every route that produces AA.

Think of a tree: the first branches are the cause, the second the evidence. Multiply along branches, add the branches ending in the evidence, then divide.

  1. 1

    Branches from B and B′

    First branches are the cause

  2. 2

    Multiply along each branch

    Gives each route to A

  3. 3

    Add the routes ending in A

    This is P(A)

  4. 4

    Divide the route through B by P(A)

    This is P(B | A)

Bayes' theorem on a tree diagram
  • NumeratorP(B)P(A∣B)P(B)P(A\mid B)
  • Denominator, P(A)P(B)P(A∣B)+P(B′)P(A∣B′)P(B)P(A\mid B)+P(B')P(A\mid B')

Worked example

2% have a condition CC. P(+∣C)=0.95P(+\mid C)=0.95 and P(+∣C′)=0.10P(+\mid C')=0.10. Find P(C∣+)P(C\mid +).

Why is the denominator P(A)P(A) in Bayes' theorem?

Three events

Add all three routes in the denominator; the events must be a partition.

If B1B_1, B2B_2, B3B_3 are mutually exclusive and together cover every outcome (a partition), the denominator is the sum of all three routes:

P(Bi∣A)=P(Bi)P(A∣Bi)∑j=13P(Bj)P(A∣Bj).P(B_i\mid A)=\frac{P(B_i)P(A\mid B_i)}{\sum_{j=1}^{3}P(B_j)P(A\mid B_j)}.

Example: machines A, B, C make 50%, 30%, 20% of bolts with defect rates 2%, 3%, 5%. P(D)=0.010+0.009+0.010=0.029P(D)=0.010+0.009+0.010=0.029, so P(C∣D)=0.0100.029=1029P(C\mid D)=\frac{0.010}{0.029}=\frac{10}{29}.

Machine C makes only 20% of the bolts but about 34% of the defective ones.

01020304050ABCMachinePercentage
  • Share of all bolts
  • Share of defective bolts
Share of all bolts made and share of defective bolts, by machine (%)

In the machine example, what is P(A | D)?

Unknowns and complements

Write the law of total probability as an equation and solve for the unknown.

Some questions give P(A)P(A) and ask for a missing conditional probability first. If P(R)=0.3P(R)=0.3, P(L∣R)=0.4P(L\mid R)=0.4, P(L∣R′)=pP(L\mid R')=p and P(L)=0.19P(L)=0.19, then 0.12+0.7p=0.190.12+0.7p=0.19, so p=0.1p=0.1.

For the evidence not happening, use the complementary branches: P(L′∣R)=0.6P(L'\mid R)=0.6 and P(L′)=1−0.19=0.81P(L')=1-0.19=0.81, so P(R∣L′)=0.180.81=29P(R\mid L')=\frac{0.18}{0.81}=\frac29.

Check your answers: every posterior lies between 0 and 1, and the posteriors for a partition add up to 1.

00.050.10.150.2Through RThrough R′Total P(L)RouteProbability
Routes to L in the example: 0.3 × 0.4 = 0.12 through R and 0.7 × 0.1 = 0.07 through R′, so P(L) = 0.19
  • Law of total probabilityP(L)=P(R)P(L∣R)+P(R′)P(L∣R′)P(L)=P(R)P(L\mid R)+P(R')P(L\mid R')

Worked example

P(R)=0.3P(R)=0.3, P(L∣R)=0.4P(L\mid R)=0.4 and P(L)=0.19P(L)=0.19. Find P(L∣R′)P(L\mid R').

With the values above, what is P(L′)P(L')?

Interpreting the answer

Rare conditions give many false positives, so most positives are from people without the condition.

Screening for a rare condition gives many false positives: although the test is accurate, most people tested do not have the condition, so false positives outnumber true positives. With 2% prevalence, only about 16% of positives actually have it.

A second, independent piece of evidence updates the probability again: for a scam filter, P(S∣F∩W)=(0.1)(0.7)(0.9)0.0756=56P(S\mid F\cap W)=\frac{(0.1)(0.7)(0.9)}{0.0756}=\frac56, up from 722\frac{7}{22} after one check.

IB questions often ask you to comment: say what the number means for the person or the system.

020406080100True positivesFalse positivesType of positive resultNumber of people
Positive results among 1000 people tested (2% have the condition)

Why do most positives come from people without the condition in the chart above?

Try an exam question

2% of a population have a condition CC. A test gives a positive result (+)(+) with probability 0.95 for people with CC and 0.10 for people without CC. (a) Find P(+)P(+). (b) Find the probability that a person with a positive result has CC.

[4 marks]

That's the notes covered.

Carry on to the next subtopic.

Try a question

Find the probability that the test is positive.

1 mark, marked against the mark scheme

Revision notes