Language of hypothesis testingAQA A-Level Maths: Revision notes

A hypothesis test uses a sample to decide whether a claim about a population parameter is believable. For a binomial model $X\sim B(n,p)$ the parameter is $p$, the probability of success on each trial.

• The null hypothesis, $H_0$, is the claim assumed true for the test. It gives one value: $H_0:p=0.6$.
• The alternative hypothesis, $H_1$, says what you suspect instead.
  • A 1-tail test has $H_1:p<p_0$ or $H_1:p>p_0$ because the suspicion has a direction.
  • A 2-tail test has $H_1:p\ne p_0$ because the suspicion is only that the value has changed. Always define $p$ in words, for example "$p$ is the probability that a seed germinates". The direction of $H_1$ comes from the wording of the question, never from the sample result.

The test statistic is the quantity calculated from the sample, here the number of successes $X$. The test works by asking: if $H_0$ is true, how likely is a result like the one observed? So you work under $H_0$ and use its distribution, for example $X\sim B(20,0.6)$ if $n=20$ and $H_0:p=0.6$. Only the value of $p$ from $H_0$ is used in the calculation.

The significance level, $\alpha$, is the probability of rejecting $H_0$ when $H_0$ is true; it is a measure of how unlikely a result must be before you reject. It is set in advance, commonly 5% or 1%. The critical region is the set of values of the test statistic that lead to rejecting $H_0$. Its boundary is the critical value. The acceptance region is the remaining values, where $H_0$ is not rejected. Finding it, for $X\sim B(20,0.6)$ with $H_1:p<0.6$ at 5%: $P(X\le7)=0.0210\le0.05$ but $P(X\le8)=0.0565>0.05$. The critical region is $X\le7$, the critical value is $7$ and the acceptance region is $X\ge8$. For a 1-tail test all of $\alpha$ is in one tail. For a 2-tail test split it: for 5%, put at most $2.5\%$ in each tail.

The $p$-value is the probability, assuming $H_0$ is true, of obtaining a result at least as extreme as the one observed. For $X\sim B(15,0.3)$ and an observed value of 9 with $H_1:p>0.3$, $p$-value $=P(X\ge9)=0.0152$. Decision rule: if the $p$-value is less than or equal to the significance level, reject $H_0$; otherwise do not reject. In a 2-tail test either compare the one-tail probability with $\alpha/2$, or double it and compare with $\alpha$.

With a discrete distribution the critical region rarely uses up exactly $\alpha$. The actual significance level is the real probability of falling in the critical region when $H_0$ is true. For the 2-tail test $X\sim B(30,0.35)$ with critical region $X\le5$ or $X\ge17$: $0.0233+0.0124=0.0357$, which is below 5%. Conclusions must be in context and cautious. If the result is in the critical region: "There is sufficient evidence at the 5% level to suggest that the proportion has changed." If not: "There is insufficient evidence to suggest that it has changed." Never say that $H_0$ has been proved. The significance level is also the probability of wrongly rejecting a true $H_0$.