4.11 Hypothesis testing: chi-squared and t-testsIB Maths: Applications and Interpretation SL: Revision notes

A hypothesis test decides whether sample data give enough evidence against a claim.

• The null hypothesis $H_0$ is the claim being tested, written as an equation, such as $\mu_A=\mu_B$, or in words (for example 'the variables are independent').
• The alternative hypothesis $H_1$ is what we accept if there is enough evidence against $H_0$. For a mean it may be $\neq$ (two-tailed) or $>$ or $<$ (one-tailed).
• The significance level is the probability of rejecting $H_0$ when it is true. Common levels are $1\%$, $5\%$ and $10\%$.
• The $p$-value is the probability of getting a result at least as extreme as the one observed, if $H_0$ is true. Decision rule: if $p<$ significance level, reject $H_0$; if $p>$ significance level, do not reject $H_0$. State the conclusion in context. Say 'there is evidence that…' or 'there is not enough evidence that…'. Never say that $H_0$ has been proved.

The $\chi^2$ test for independence tests whether two categorical variables are associated. $H_0$: the variables are independent. $H_1$: they are not independent (associated). Data are in a contingency table of observed frequencies $O$. The expected frequency in each cell, if $H_0$ is true, is $E=\frac{\text{row total}\times\text{column total}}{\text{grand total}}.$ The statistic is $\chi^2=\sum\frac{(O-E)^2}{E}$ and the degrees of freedom are $\nu=(\text{rows}-1)(\text{columns}-1)$. In examinations tables have at most $4$ rows or columns, $\nu>1$, and you use your GDC for $\chi^2$ and the $p$-value. Only upper-tail tests are used: reject $H_0$ if $\chi^2$ is greater than the critical value (given), or if $p<$ the significance level. Example: Year $10$ travel $20,30,25$ and Year $11$ travel $30,25,20$ (walk, bus, car). The expected values are $25,27.5,22.5$ in both rows, $\nu=2$, $\chi^2=3.01$ and $p=0.222>0.05$, so there is no evidence of an association.

The $\chi^2$ goodness of fit test tests whether data fit a stated distribution. $H_0$: the data fit the distribution (for example, equally likely categories). $H_1$: they do not. Find expected frequencies from the distribution: total $\times$ probability of each category. For $n$ categories, at SL the degrees of freedom are $\nu=n-1$. Then $\chi^2=\sum\frac{(O-E)^2}{E}$ and compare with the critical value, or compare $p$ with the significance level. Example: $200$ customers choose among $4$ desserts: $68,52,44,36$. If the choices are equally likely, each expected frequency is $\frac{200}{4}=50$. $\chi^2=\frac{18^2+2^2+(-6)^2+(-14)^2}{50}=11.2$ and $\nu=3$. At $5\%$ the critical value is $7.81$. Since $11.2>7.81$ (and $p=0.0107<0.05$), reject $H_0$: there is evidence that the desserts are not chosen equally often.

The $t$-test uses the sample means to test whether the means of two populations differ. In examinations the samples are unpaired, the population variance is unknown and you assume equal variances, so the pooled two-sample $t$-test is used on your GDC. The test needs the data in both populations to be normally distributed. Hypotheses: $H_0:\mu_A=\mu_B$ and $H_1:\mu_A\neq\mu_B$ (two-tailed), or $H_1:\mu_A>\mu_B$ (one-tailed). Enter the sample means, sample standard deviations and sample sizes (or the raw data) into the GDC and choose the correct alternative. The GDC returns $t$ and the $p$-value for the alternative chosen. Compare $p$ with the significance level. Example: class A ($12$ students, mean $68.4$, standard deviation $7.2$) and class B ($10$ students, mean $62.1$, standard deviation $8.1$). One-tailed test, $H_1:\mu_A>\mu_B$: $p=0.0339<0.05$, reject $H_0$. Two-tailed: $p=0.0677>0.05$, do not reject $H_0$. The one-tailed $p$-value is half the two-tailed value when the sample difference is in the direction of $H_1$.

Write conclusions in context and compare $p$ with the given significance level: the same $p$ can be significant at $10\%$ but not at $5\%$ (for example $p=0.0677$).

• A significant result is evidence of an effect, not proof, and it does not say how large or important the effect is.
• For $\chi^2$ tests, expected frequencies of $5$ or less make the test unreliable, so categories may need combining.
• The $t$-test needs the populations to be normal with equal variances; if not, the conclusion may be unreliable.
• A larger sample makes it easier to detect a real difference. Do not use a two-tailed test when only one direction was predicted in advance, and do not change the significance level after seeing the result.