Estimators, bias and the sampling distribution of the mean

Estimators, bias and the sampling distribution of the mean

Spec 3.1–2 · Edexcel International A Level Further Maths: Revision notes

Studying this course? Keep your progress in one place.

Key facts

  • Bias is E(T)−θ\mathrm{E}(T)-\theta; unbiased means E(T)=θ\mathrm{E}(T)=\theta.
  • An estimator is a statistic used to estimate a parameter; an estimate is its value for one sample.
  • Unbiased estimates: xˉ=∑xn\bar x=\frac{\sum x}{n} and s2=1n−1(∑x2−nxˉ2)s^2=\frac{1}{n-1}\left(\sum x^2-n\bar x^2\right).
  • E(Xˉ)=μ\mathrm{E}(\bar X)=\mu and Var(Xˉ)=σ2n\mathrm{Var}(\bar X)=\frac{\sigma^2}{n}; the standard error is σn\frac{\sigma}{\sqrt n}.

Estimators and estimates

An estimator is a random variable used to estimate a parameter; an estimate is a number.

A parameter is a fixed but usually unknown number describing a population, such as μ\mu or σ2\sigma^2. A statistic is a quantity calculated from a random sample that involves no unknown parameter, such as Xˉ=1n∑Xi\bar X=\frac1n\sum X_i.

An estimator is a statistic used to estimate a parameter; an estimate is the number it takes for one sample. Xˉ\bar X is an estimator of μ\mu, and xˉ=12.5\bar x=12.5 is an estimate.

  1. 1

    Population

    Parameter μ: fixed, usually unknown

  2. 2

    Random sample

    X₁, …, Xₙ

  3. 3

    Estimator X̄

    A statistic: a random variable

  4. 4

    Estimate x̄ = 12.5

    The number for one sample

From population to estimate.

Parameter

  • A fixed number for the population
  • Usually unknown: μ, σ²

Statistic

  • Calculated from a sample
  • A random variable: X̄

Estimator

  • A statistic used to estimate
  • Capital letter: X̄

Estimate

  • Its value for one sample
  • Lower case: x̄ = 12.5

A sample gives xˉ=12.5\bar x=12.5. Is 12.512.5 an estimator or an estimate of μ\mu?

Bias

An estimator is unbiased if its expected value equals the parameter.

For an estimator TT of a parameter θ\theta, the bias is E(T)−θ\mathrm{E}(T)-\theta. If E(T)=θ\mathrm{E}(T)=\theta, then TT is an unbiased estimator: on average, over many samples, it gives the true value.

To test a linear estimator use E(aX1+bX2)=aE(X1)+bE(X2)\mathrm{E}(aX_1+bX_2)=a\mathrm{E}(X_1)+b\mathrm{E}(X_2).

1234567890.20.40.60.811.2xyθ = 5Unbiased: E(T) = θBiased: E(T) < θ
Sampling distributions of two estimators of θ = 5 (illustrative): the unbiased one is centred on θ; the biased one is centred on E(T) = 4.
  • BiasE(T)−θ\mathrm{E}(T)-\theta
  • UnbiasedE(T)=θ\mathrm{E}(T)=\theta

Worked example

X1,X2,X3X_1,X_2,X_3 are independent with mean μ\mu. Show that T=X1+2X23T=\frac{X_1+2X_2}{3} is unbiased, and find the bias of U=X1+X2+X32U=\frac{X_1+X_2+X_3}{2}.

E(T)=2μ\mathrm{E}(T)=2\mu for an estimator TT of μ\mu. What is the bias?

Unbiased estimates

Use xˉ\bar x for the mean and divide by n−1n-1 for the variance.

From a random sample x1,…,xnx_1,\dots,x_n, xˉ=∑xn\bar x=\frac{\sum x}{n} is an unbiased estimate of μ\mu and the sample variance s2=1n−1∑(xi−xˉ)2s^2=\frac{1}{n-1}\sum(x_i-\bar x)^2 is an unbiased estimate of σ2\sigma^2.

Dividing by nn is biased: its expected value is n−1nσ2\frac{n-1}{n}\sigma^2, which underestimates σ2\sigma^2. No proofs are required.

00.20.40.60.81True σ²Divide by nDivide by n − 1Estimator of σ²Expected value
With n = 5 and true σ² = 1, dividing by n has expected value 0.8; dividing by n − 1 has expected value 1.
  • Unbiased estimate of μxˉ=∑xn\bar x=\dfrac{\sum x}{n}
  • Unbiased estimate of σ²s2=∑x2−nxˉ2n−1s^2=\dfrac{\sum x^2-n\bar x^2}{n-1}

Worked example

A sample has n=8n=8, ∑x=100\sum x=100 and ∑x2=1292\sum x^2=1292. Find unbiased estimates of the mean and variance.

A sample has n=5n=5 and ∑(xi−xˉ)2=20\sum(x_i-\bar x)^2=20. What is the unbiased estimate of the variance?

Distribution of the sample mean

Xˉ\bar X is centred on μ\mu with variance σ2n\frac{\sigma^2}{n}, so it is narrower than one observation.

If X1,…,XnX_1,\dots,X_n is a random sample from a population with mean μ\mu and variance σ2\sigma^2, then E(Xˉ)=μ\mathrm{E}(\bar X)=\mu and Var(Xˉ)=σ2n\mathrm{Var}(\bar X)=\frac{\sigma^2}{n}. The standard deviation σn\frac{\sigma}{\sqrt n} is the standard error.

If the population is Normal, then exactly Xˉ∼N(μ,σ2n)\bar X\sim\mathrm{N}\left(\mu,\frac{\sigma^2}{n}\right). This is the sampling distribution of the mean. When σ\sigma is unknown the standard error is estimated by sn\frac{s}{\sqrt n}.

1001201401601802000.020.040.060.08xysame centreX, N(150, 20²)X̄, N(150, 5²)
One observation X∼N(150,202)X\sim\mathrm{N}(150,20^2) (dashed) and the mean of 1616: Xˉ∼N(150,52)\bar X\sim\mathrm{N}(150,5^2).

X∼N(50,36)X\sim\mathrm{N}(50,36) and n=9n=9. What is the standard error of Xˉ\bar X?

Using the sample mean

State Xˉ∼N(μ,σ2/n)\bar X\sim\mathrm{N}(\mu,\sigma^2/n) and standardise with the standard error.

Standardise with Z=Xˉ−μσ/n∼N(0,1)Z=\frac{\bar X-\mu}{\sigma/\sqrt n}\sim\mathrm{N}(0,1). Because the standard error is σn\frac{\sigma}{\sqrt n}, it halves when the sample size is multiplied by 44, so larger samples give more reliable estimates.

102030405060246810xyn = 4: 5n = 16: 2.5SE = 10/√n
Standard error σ/√n against n for σ = 10: multiplying n by 4 (4 to 16) halves it (5 to 2.5).
  • Standard errorσn\dfrac{\sigma}{\sqrt n}
  • n × 4standard error halves

Worked example

Apple masses are N(150,202)\mathrm{N}(150,20^2) and n=16n=16. Find P(Xˉ>156)\mathrm{P}(\bar X>156), and compare P(∣Xˉ−150∣<4)\mathrm{P}(|\bar X-150|<4) with P(∣X−150∣<4)\mathrm{P}(|X-150|<4) for one apple.

The sample size is multiplied by 44. What happens to the standard error?

Try an exam question

A random sample of 1010 observations has ∑x=125\sum x=125 and ∑x2=1665\sum x^2=1665. Find unbiased estimates of the population mean and variance.

[4 marks]

That's the notes covered.

Carry on to the next subtopic.

Try a question

What is an unbiased estimate of the population mean?

1 mark, marked against the mark scheme

Revision notes