Type I and Type II errorsAQA A-Level Further Maths: Revision notes

Section 1

Type I and Type II errors

A hypothesis test is a decision based on sample evidence, so it can be wrong.

A Type I error is rejecting $\mathrm{H}_0$ when it is true.
A Type II error is failing to reject $\mathrm{H}_0$ when it is false.

In this topic you must be able to define each in the context of the question and calculate the probability of a Type I error.

Key termsType I errorType II error

Common mistake

Describing the error in general terms ('rejecting $\mathrm{H}_0$ when it is true'). Give it in context: say what is wrongly concluded and what is really true.

Section 2

Describing errors in context

Write what the test would conclude and what is actually true.

Example: a seed company claims $p=0.8$ and a gardener tests $\mathrm{H}_1:p<0.8$ .

Type I error: concluding that the probability of germination is below $0.8$ when it is in fact $0.8$ .
Type II error: concluding there is no evidence the probability is below $0.8$ when it is in fact lower.

Key termsin context

Section 3

Probability of a Type I error: binomial tests

The probability of a Type I error is the probability that the test statistic falls in the critical region when $\mathrm{H}_0$ is true. This is also the actual significance level of the test.

Example: $\mathrm{H}_0:p=0.3$ against $\mathrm{H}_1:p>0.3$ with $X\sim\mathrm{B}(20,p)$ and critical region $X\ge10$ . With $p=0.3$ , $\mathrm{P}(\text{Type I error})=\mathrm{P}(X\ge10)=1-\mathrm{P}(X\le9)=1-0.9520=0.0480.$ Always use the value of the parameter given by $\mathrm{H}_0$ .

Key termscritical regionactual significance level

Common mistake

Using the wrong boundary, for example $\mathrm{P}(X\ge9)$ instead of $\mathrm{P}(X\ge10)$ when the critical region is $X\ge10$ .

Section 4

Probability of a Type I error: Poisson tests

The method is the same with the Poisson distribution.

Example: $\mathrm{H}_0:\lambda=6$ , $\mathrm{H}_1:\lambda<6$ , reject $\mathrm{H}_0$ if $Y\le2$ . With $Y\sim\mathrm{Po}(6)$ : $\mathrm{P}(\text{Type I error})=\mathrm{P}(Y\le2)=e^{-6}(1+6+18)=0.0620.$ For a two-tailed critical region add both tails. For $\lambda=4$ and critical region $X=0$ or $X\ge9$ : $\mathrm{P}=e^{-4}+\left[1-\mathrm{P}(X\le8)\right]=0.0183+0.0214=0.0397.$

Key termstwo-tailed critical region

Section 5

Choosing a critical region

To carry out a test at the 5% level, take the largest critical region whose probability is at most $0.05$ .

Example: $X\sim\mathrm{B}(15,0.8)$ and $\mathrm{H}_1:p<0.8$ . $\mathrm{P}(X\le9)=0.0611>0.05$ but $\mathrm{P}(X\le8)=0.0181\le0.05$ , so the critical region is $X\le8$ and the actual significance level is $0.0181$ .

Making the critical region larger increases the probability of a Type I error. The actual significance level can be well below the nominal 5% because the distribution is discrete.

Key termsnominal significance level

Exam tip

Test a boundary value, then the next one down, and say which satisfies the condition.

Section 6

Repeated testing

If a true $\mathrm{H}_0$ is tested several times independently, each test has probability $\alpha$ of a Type I error. The probability of at least one wrong rejection in $n$ tests is $1-(1-\alpha)^n.$ With $\alpha=0.0397$ and $n=10$ , this is $1-0.9603^{10}=0.333$ . Using many tests makes at least one false alarm quite likely.

Key termsfalse rejection

That's the notes covered.

Carry on to the next subtopic.

Exam questions on Type I and Type II errors

A manufacturer claims that 30% of its cereal boxes contain a prize. To test whether the proportion is greater than 30%, a sample of 20 boxes is taken. Let $X$ be the number of boxes in the sample that contain a prize and $p$ the proportion of all boxes that contain a prize. The test is $\mathrm{H}_0:p=0.3$ against $\mathrm{H}_1:p>0.3$ , and $\mathrm{H}_0$ is rejected if $X\ge10$ .
Find the significance level of the test if the critical region were changed to $X\ge9$ .2 marks
The number of calls received by a call centre in an hour is modelled by $Y\sim\mathrm{Po}(\lambda)$ . To test $\mathrm{H}_0:\lambda=6$ against $\mathrm{H}_1:\lambda<6$ , the manager counts the calls in one hour and rejects $\mathrm{H}_0$ if $Y\le2$ .
The manager wants the probability of a Type I error to be at most 5%. Find the largest critical region of the form $Y\le c$ that she can use.2 marks
A seed company claims that the probability that a seed germinates is $0.8$ . A gardener plants 15 seeds and suspects that the true probability is lower. Let $X$ be the number of seeds that germinate and $p$ the probability that a seed germinates. She tests $\mathrm{H}_0:p=0.8$ against $\mathrm{H}_1:p<0.8$ at the 5% significance level, using $X\sim\mathrm{B}(15,p)$ .
Find the critical region for the test.3 marks

See the full worksheet

Written by the Exaim team, led by Shaun Daswani (Head of Upper Secondary, Improve ME Institute; MSc Financial Mathematics, Imperial College London; BSc, UCL) and Jason Daswani (operational lead, Improve ME Institute; LSE).