4.18 Hypothesis testing: means, proportions, correlation and errorsIB Maths: Applications and Interpretation HL: Revision notes

A hypothesis test decides whether sample data give enough evidence against a claim. The null hypothesis $\mathrm{H}_0$ is the claim to be tested (it states a parameter equals a value, for example $\mu=500$ or $p=0.3$). The alternative hypothesis $\mathrm{H}_1$ says what we suspect: $\ne$ (two-tailed) or $<$ / $>$ (one-tailed). Hypotheses are about the population parameter, never the sample statistic. The significance level $\alpha$ (for example $5\%$) is the probability of rejecting $\mathrm{H}_0$ when it is true. The $p$-value is the probability, assuming $\mathrm{H}_0$ is true, of a result at least as extreme as the one observed. If $p<\alpha$, reject $\mathrm{H}_0$; otherwise do not reject. For a two-tailed test compare $p$ with $\alpha$ after doubling the tail probability (or compare each tail with $\frac{\alpha}{2}$). The critical region is the set of values of the test statistic that lead to rejecting $\mathrm{H}_0$; its boundary is the critical value. Conclusions must be in context and cautious: 'sufficient evidence to reject $\mathrm{H}_0$' or 'insufficient evidence to reject $\mathrm{H}_0$', never 'proved'.

$\sigma$ known (normal population, or large $n$): under $\mathrm{H}_0$, $\bar{X}\sim N\left(\mu_0,\frac{\sigma^2}{n}\right)$, so use the $z$-test; the GDC gives the $p$-value. Example: $\mu_0=500$, $\sigma=8$, $n=36$, $\bar{x}=497.4$, $\mathrm{H}_1:\mu<500$: $z=\frac{497.4-500}{8/6}=-1.95$ and $p=0.0256<0.05$, so reject $\mathrm{H}_0$. $\sigma$ unknown: use the one-sample $t$-test with $n-1$ degrees of freedom, whatever the sample size. The GDC $t$-test takes the data (or $\bar{x}$, $s_{n-1}$, $n$) and gives $t$ and the $p$-value; you are not expected to calculate critical regions for $t$-tests. Paired data: samples may be paired (matched pairs). Take the differences $d=x_1-x_2$ and treat them as a single sample, testing $\mathrm{H}_0:\mu_d=0$. Unpaired samples are two independent samples.

Proportion (binomial): $\mathrm{H}_0:p=p_0$. With $n$ trials, $X\sim\mathrm{B}(n,p_0)$ under $\mathrm{H}_0$. For $\mathrm{H}_1:p>p_0$ the $p$-value for observing $x$ is $\mathrm{P}(X\ge x)=1-\mathrm{P}(X\le x-1)$. Example: $n=25$, $p_0=0.3$, $x=12$: $p=0.0442<0.05$, so reject $\mathrm{H}_0$. Poisson mean: $\mathrm{H}_0:\lambda=\lambda_0$, and $X\sim\mathrm{Po}(\lambda_0)$ under $\mathrm{H}_0$. Rescale $\lambda$ to the period observed. Binomial and Poisson tests are one-tailed only in IB. The critical region for a discrete variable is found by testing values: choose the region with the largest probability that is still less than the significance level. Example: $X\sim\mathrm{B}(20,0.7)$, $\mathrm{H}_1:p<0.7$: $\mathrm{P}(X\le10)=0.0480<0.05$ but $\mathrm{P}(X\le11)=0.113>0.05$, so the critical region is $X\le10$.

A Type I error is rejecting $\mathrm{H}_0$ when it is true; $\mathrm{P}(\text{Type I})$ is the probability that the test statistic lies in the critical region under $\mathrm{H}_0$. For continuous tests this equals $\alpha$; for discrete tests it is the actual probability of the critical region, which may be below $\alpha$ (for example $0.0480$). A Type II error is not rejecting $\mathrm{H}_0$ when it is false. Its probability is found under a specific true value of the parameter: $\mathrm{P}(\text{test statistic not in the critical region}\mid\text{true value})$. Examples: Normal, $\sigma=8$, $n=36$, $\mathrm{H}_0:\mu=500$, $\mathrm{H}_1:\mu<500$ at $5\%$: critical region $\bar{x}<497.8$. If the true $\mu=497$, $\mathrm{P}(\text{Type II})=\mathrm{P}(\bar{X}\ge497.8\mid\mu=497)=0.273$. Binomial: $\mathrm{B}(20,0.7)$, critical region $X\le10$; if $p=0.5$, $\mathrm{P}(\text{Type II})=\mathrm{P}(X\ge11)=0.412$. Poisson: $\mathrm{Po}(6)$, critical region $X\ge11$; if $\lambda=9$, $\mathrm{P}(\text{Type II})=\mathrm{P}(X\le10)=0.706$.

For bivariate data from a bivariate normal population, the sample product-moment correlation coefficient $r$ estimates the population coefficient $\rho$. To test whether there is a linear correlation: $\mathrm{H}_0:\rho=0$ against $\mathrm{H}_1:\rho\ne0$ (or $\rho>0$, $\rho<0$). Use technology; in examinations the data (or the output) will be given. If the $p$-value is less than the significance level, reject $\mathrm{H}_0$: there is sufficient evidence of a linear correlation. Example: $r=0.78$ with $p=0.004$ at $5\%$ gives evidence that $\rho\ne0$. Correlation does not prove causation.