Hypothesis testing conceptsEdexcel International A Level Further Maths: Revision notes

A hypothesis test uses sample data to decide whether a claim about a population parameter is believable. We assume the claim is true, work out how likely the observed result would be, and reject the claim only if the result would be very unlikely. A test can show evidence for or against a claim, but it never proves a hypothesis true or false. The conclusion is always worded as sufficient or insufficient evidence.

The null hypothesis, $\mathrm{H}_0$, is the claim assumed to be true, and it always gives a single value of the parameter, for example $\mathrm{H}_0:p=0.5$ or $\mathrm{H}_0:\lambda=4$. The alternative hypothesis, $\mathrm{H}_1$, says how the parameter differs, for example $p>0.5$, $p<0.5$ or $p\ne0.5$. Both hypotheses are statements about a parameter, so write them using $p$ or $\lambda$, never about the test statistic or the sample. Define the parameter in context, such as 'let $p$ be the probability that the coin lands heads'.

The test statistic is the quantity calculated from the sample, for example the number of heads $X$. Under $\mathrm{H}_0$ its distribution is known, such as $X\sim\mathrm{B}(20,0.5)$. The critical region is the set of values of the test statistic for which $\mathrm{H}_0$ is rejected. Its edge is the critical value. The significance level is the probability of the test statistic falling in the critical region when $\mathrm{H}_0$ is true; common levels are $5\%$, $1\%$ and $10\%$. Because the test statistic is discrete, the largest critical region with probability at most the significance level is used. The actual significance level is then the true probability of being in that region, which is usually a little smaller than the stated level.

In a one-tailed test the alternative hypothesis has a single direction, $\mathrm{H}_1:p>p_0$ or $\mathrm{H}_1:p<p_0$, and all of the significance level is in one tail. Use it when the claim suggests a specific direction, such as 'the rate has fallen' or 'biased towards heads'. In a two-tailed test the alternative is $\mathrm{H}_1:p\ne p_0$, used when any difference matters, such as 'the mean has changed'. The significance level is split equally, so a $5\%$ test has $2.5\%$ in each tail. Example: $X\sim\mathrm{B}(20,0.5)$, two-tailed at $5\%$. $\mathrm{P}(X\le5)=0.0207<0.025$ but $\mathrm{P}(X\le6)=0.0577>0.025$, so the critical region is $X\le5$ or $X\ge15$. The actual significance level is $2\times0.0207=0.0414$.

Decide by comparing the observed value with the critical region, or the probability with the significance level. If the observed value is in the critical region, reject $\mathrm{H}_0$; otherwise do not reject it. Then write a conclusion in context that is not over-confident. For example: 'There is sufficient evidence, at the $5\%$ level, that the coin is biased towards heads.' or 'There is insufficient evidence that the pupil is not just guessing.' The same data can lead to different conclusions at different significance levels: a larger level makes it easier to reject $\mathrm{H}_0$.

A model such as 'late arrivals are Poisson with mean $4$' is a claim about a parameter, so it can be tested with new data. If the data give a result in the critical region, the model no longer fits and should be refined, for example by changing the mean. If $\mathrm{H}_0$ is not rejected, there is no evidence against the model and it may be kept, although it has not been proved correct.