Type II errors and powerAQA A-Level Further Maths: Revision notes

A Type II error is failing to reject $\mathrm{H}_0$ when $\mathrm{H}_0$ is false.

To find its probability you need a specific true value of the parameter. Then $\mathrm{P}(\text{Type II error})=\mathrm{P}(\text{test statistic is outside the critical region}\mid\text{true value}).$ The distribution used is the one for the true value, not the value in $\mathrm{H}_0$. (For a Type I error it is the other way round.)

The power of a test is the probability of rejecting $\mathrm{H}_0$ when it is false: $\text{power}=1-\mathrm{P}(\text{Type II error})=\mathrm{P}(\text{test statistic in the critical region}\mid\text{true value}).$ Power depends on the true value, so a test has a different power for each possible alternative. A high power means the test is good at detecting a real change.

Binomial. $\mathrm{H}_0:p=0.5$, $\mathrm{H}_1:p>0.5$, $X\sim\mathrm{B}(10,p)$, reject if $X\ge8$. True $p=0.7$: $\mathrm{P}(\text{Type II})=\mathrm{P}(X\le7)=0.6172,\qquad\text{power}=\mathrm{P}(X\ge8)=0.3828.$ Poisson. $\mathrm{H}_0:\lambda=5$, $\mathrm{H}_1:\lambda>5$, reject if $Y\ge10$. True $\lambda=8$: $\mathrm{P}(\text{Type II})=\mathrm{P}(Y\le9)=0.7166,\qquad\text{power}=0.2834.$ Always write the distribution with the true parameter first.

For a test of a mean with $\sigma$ known, $\bar x\sim\mathrm{N}\left(\mu,\frac{\sigma^2}{n}\right)$ with $\mu$ the true mean.

One-tailed: $\mathrm{H}_0:\mu=500$, $\mathrm{H}_1:\mu<500$, $\sigma=4$, $n=16$, reject if $\bar x<498$. True mean $497$: $\bar x\sim\mathrm{N}(497,1^2)$, so $\text{power}=\mathrm{P}(\bar x<498)=\mathrm{P}(Z<1)=0.841.$ Two-tailed: reject if $\bar x<993$ or $\bar x>1007$, with true mean $1010$ and standard deviation $4$: $\mathrm{P}(\text{Type II})=\mathrm{P}(993\le\bar x\le1007)=\mathrm{P}(-4.25\le Z\le-0.75)=0.227.$

Power is higher when:

• the true value is further from the $\mathrm{H}_0$ value, because a real difference is easier to detect,
• the sample size is larger, because $\bar x$ has a smaller standard deviation,
• the critical region is larger (a larger significance level), but this also raises the probability of a Type I error.

So there is a trade-off: enlarging the critical region lowers $\mathrm{P}$(Type II) but raises $\mathrm{P}$(Type I). Only a bigger sample can improve both.