Hypothesis tests for the difference between two meansEdexcel International A Level Further Maths: Revision notes

Two independent random samples are taken, one from each of two populations, to test whether their means differ. Let $X$ have mean $\mu_x$ and variance $\sigma_x^2$, with sample size $n_x$ and sample mean $\bar X$, and similarly $Y$ with $\mu_y$, $\sigma_y^2$, $n_y$ and $\bar Y$. The hypotheses are $H_0:\mu_x=\mu_y$ (or $\mu_x-\mu_y=0$) against $H_1:\mu_x\neq\mu_y$ (two-tailed), or $H_1:\mu_x>\mu_y$ or $H_1:\mu_x<\mu_y$ (one-tailed). As always, the null hypothesis holds the equality and the hypotheses concern population means, not sample means. The statistic that is tested is the difference of the sample means, $\bar X-\bar Y$.

If $X\sim N(\mu_x,\sigma_x^2)$ and $Y\sim N(\mu_y,\sigma_y^2)$ are independent, then $\bar X\sim N\left(\mu_x,\frac{\sigma_x^2}{n_x}\right)$ and $\bar Y\sim N\left(\mu_y,\frac{\sigma_y^2}{n_y}\right)$, so $\bar X-\bar Y\sim N\left(\mu_x-\mu_y,\ \frac{\sigma_x^2}{n_x}+\frac{\sigma_y^2}{n_y}\right).$ The means subtract but the variances add, because subtracting independent random variables does not remove any of the variability. The standard error of the difference is $\sqrt{\frac{\sigma_x^2}{n_x}+\frac{\sigma_y^2}{n_y}}$. Example: $\sigma_x=4$, $n_x=20$, $\sigma_y=5$, $n_y=25$ gives variance $\frac{16}{20}+\frac{25}{25}=1.8$.

Under $H_0$ ($\mu_x=\mu_y$), the test statistic is $Z=\frac{(\bar X-\bar Y)-(\mu_x-\mu_y)}{\sqrt{\frac{\sigma_x^2}{n_x}+\frac{\sigma_y^2}{n_y}}}=\frac{\bar X-\bar Y}{\sqrt{\frac{\sigma_x^2}{n_x}+\frac{\sigma_y^2}{n_y}}}\sim N(0,1).$ Compare $z$ with the critical value for the significance level and the tail(s): $\pm1.96$ (two-tailed $5\%$), $1.645$ or $-1.645$ (one-tailed $5\%$), $2.326$ (one-tailed $1\%$), $\pm2.576$ (two-tailed $1\%$). Worked example: fertilisers $A$ and $B$, $\sigma_A=2.1$, $\sigma_B=2.4$, $n_A=30$, $n_B=40$, $\bar x_A=14.6$, $\bar x_B=13.8$, $H_1:\mu_A>\mu_B$. The variance of the difference is $0.147+0.144=0.291$, so $z=\frac{0.8}{0.539}=1.48<1.645$. Do not reject $H_0$: insufficient evidence at the $5\%$ level that $A$ gives taller seedlings.

When the population variances are not known, use the unbiased estimates $s_x^2$ and $s_y^2$. If both samples are large, the Central Limit Theorem makes $\bar X$ and $\bar Y$ approximately Normal (even if the populations are not Normal), and $S_x^2$, $S_y^2$ are close enough to $\sigma_x^2$, $\sigma_y^2$ that $Z=\frac{(\bar X-\bar Y)-(\mu_x-\mu_y)}{\sqrt{\frac{S_x^2}{n_x}+\frac{S_y^2}{n_y}}}\approx N(0,1).$ Example: stores $P$ and $Q$ with $n_P=60$, $s_P^2=81$, $n_Q=50$, $s_Q^2=64$, $\bar x_P=42.5$, $\bar x_Q=39.8$. The variance is $\frac{81}{60}+\frac{64}{50}=2.63$ and $z=\frac{2.7}{1.622}=1.66<1.96$, so there is insufficient evidence of a difference at the $5\%$ level. For small samples this approximation is not reliable, and a different distribution would be needed, which is not required here. Always state that the samples are large.

Give the conclusion in context and at the stated level: 'there is evidence that method $A$ gives a shorter mean time than method $B$', or 'there is insufficient evidence that the mean masses differ'. Never claim that $H_0$ is proved. Assumptions to state or check: the samples are random and independent; for small samples the populations are Normal (and the variances known); for large samples the Central Limit Theorem is used. A test about means says nothing about every individual: some members of the population with the smaller mean may still exceed some members of the population with the larger mean. A result can be significant at $5\%$ but not at $1\%$, so the level matters. If $z=-2.37$, the test is significant in a one-tailed test at $5\%$ ($-1.645$) and at $1\%$ ($-2.326$).