Estimators, bias and the sampling distribution of the meanEdexcel International A Level Further Maths: Revision notes

A parameter is a fixed but usually unknown number describing a population, such as the mean $\mu$ or the variance $\sigma^2$. A statistic is a quantity calculated from a random sample that does not involve any unknown parameter, for example $\bar X=\frac{1}{n}\sum X_i$. Because the sample is random, a statistic varies from sample to sample and is itself a random variable with its own distribution. An estimator is a statistic used to estimate a parameter. An estimate is the numerical value the estimator takes for one particular sample. For instance $\bar X$ is an estimator of $\mu$, and $\bar x=12.5$ is an estimate. Capital letters are used for random variables (estimators) and lower case for the observed values (estimates).

For an estimator $T$ of a parameter $\theta$, the bias is $E(T)-\theta$. If $E(T)=\theta$, then $T$ is an unbiased estimator: on average, over many samples, it gives the true value. A biased estimator systematically over- or under-estimates $\theta$. To test a linear estimator, find its expected value using $E(aX_1+bX_2)=aE(X_1)+bE(X_2)$. For observations with mean $\mu$:

• $T=\frac{X_1+2X_2}{3}$ has $E(T)=\frac{\mu+2\mu}{3}=\mu$, so it is unbiased;
• $T=\frac{X_1+X_2+X_3}{2}$ has $E(T)=\frac{3\mu}{2}$, so it is biased, with bias $\frac{\mu}{2}$ (it overestimates). An estimator is not 'better' just because it is unbiased. The sample size matters too, through the standard error below.

From a random sample $x_1,\dots,x_n$ of a population with mean $\mu$ and variance $\sigma^2$: $\bar x=\frac{\sum x}{n}\ \text{ is an unbiased estimate of }\mu,\qquad s^2=\frac{1}{n-1}\sum(x_i-\bar x)^2\ \text{ is an unbiased estimate of }\sigma^2.$ For calculation, use $s^2=\frac{1}{n-1}\left(\sum x^2-n\bar x^2\right)=\frac{1}{n-1}\left(\sum x^2-\frac{(\sum x)^2}{n}\right)$. Worked example: $n=8$, $\sum x=100$, $\sum x^2=1292$. Then $\bar x=12.5$ and $s^2=\frac{1292-8(12.5)^2}{7}=\frac{42}{7}=6$. The alternative $\frac1n\sum(x_i-\bar x)^2$ is biased: its expected value is $\frac{n-1}{n}\sigma^2$, so on average it underestimates $\sigma^2$. This is why the divisor is $n-1$. (No proofs are required.)

If $X_1,\dots,X_n$ is a random sample from a population with mean $\mu$ and variance $\sigma^2$, the sample mean $\bar X$ has $E(\bar X)=\mu,\qquad \operatorname{Var}(\bar X)=\frac{\sigma^2}{n}.$ Its standard deviation, $\frac{\sigma}{\sqrt n}$, is called the standard error of the mean. If the population is itself Normal, $X\sim N(\mu,\sigma^2)$, then exactly $\bar X\sim N\left(\mu,\frac{\sigma^2}{n}\right).$ The distribution of $\bar X$ is called a sampling distribution. It is centred on $\mu$ (so $\bar X$ is unbiased), and it is narrower than the distribution of a single observation. When $\sigma$ is not known, the standard error is estimated by $\frac{s}{\sqrt n}$.

To find probabilities about $\bar X$, state its distribution and standardise with the standard error: $Z=\frac{\bar X-\mu}{\sigma/\sqrt n}\sim N(0,1).$ Worked example: apple masses are $N(150,20^2)$ and $n=16$. Then $\bar X\sim N(150,25)$ with standard error $5$, so $P(\bar X>156)=P\left(Z>\frac{156-150}{5}\right)=P(Z>1.2)=0.115.$ A single apple within $4$ g of $150$ g has probability $P(|Z|<0.2)=0.159$, but the sample mean has $P(|Z|<0.8)=0.576$: the mean of a sample is much less variable. Because the standard error is $\frac{\sigma}{\sqrt n}$, it halves when the sample size is multiplied by 4, and larger samples give more reliable estimates.