Estimators, standard error and bias
Studying this course? Keep your progress in one place.
Key facts
- An estimator is a random statistic; an estimate is the number from one sample.
- Bias . is an unbiased estimator of .
- is unbiased for ; dividing by is biased low.
- Standard error of is , estimated by .
- Prefer the unbiased estimator with the smaller variance.
Estimators and estimates
An estimator is a random variable; an estimate is the number it gives for one sample.
A population parameter such as or is a fixed, usually unknown number. An estimator is a statistic, a function of the random sample , such as . An estimate is the value it takes for one observed sample.
Because the sample is random, an estimator has a sampling distribution, whose mean and variance show how good it is.
- 1
Population parameter
fixed, unknown, e.g.
- 2
Random sample
- 3
Estimator
a statistic such as (a random variable)
- 4
Estimate
the number for one sample
Which of these is a number, not a random variable?
Bias
An estimator is unbiased when its expected value equals the parameter.
The bias of an estimator of is . is unbiased if . A positive bias means it overestimates on average; a negative bias, that it underestimates.
Know that and that is unbiased for . To test any linear estimator, use .
- Bias
- Unbiased
Worked example
Show that is an unbiased estimator of .
- 1
.
- 2
Each , so .
An estimator has . What is its bias?
Why divide by n − 1
Deviations from are slightly too small, and dividing by corrects the bias exactly.
Deviations are measured from , which is fitted to the same data, so they are slightly too small. Dividing by gives with , with bias .
- Divide by n
- Divide by n − 1
- Unbiased estimate of
- Dividing by biased low:
Worked example
, , . Find .
- 1
.
- 2
.
Dividing by gives an estimator of that is...
Standard error
The standard error is the standard deviation of the sampling distribution; for it is .
For independent observations, , so .
The standard error of is , estimated by when is unknown. A smaller standard error means a more precise estimator.
- Standard error of , estimated by
Worked example
, . Find the estimated standard error, then the value for .
- 1
: .
- 2
: .
The sample size is multiplied by . The standard error is...
Choosing between estimators
Prefer the unbiased estimator with the smaller variance, which is more efficient.
A good estimator is unbiased and has a small variance. If both are unbiased, prefer the one with the smaller variance: it is more efficient.
A biased estimator may have a smaller variance but misses the target systematically, so it is usually rejected. In an evaluation, state bias first, then variances, then a conclusion.
| A = X̄ (n = 4) | B = (X₁ + X₂)/2 | |
|---|---|---|
| Bias | Unbiased | Unbiased |
| Variance | σ²/4 | σ²/2 |
A = X̄ (n = 4)
- Bias:
- Unbiased
- Variance:
- σ²/4
B = (X₁ + X₂)/2
- Bias:
- Unbiased
- Variance:
- σ²/2
Worked example
For from , compare and .
- 1
Both are unbiased: .
- 2
and .
Two unbiased estimators of the same parameter have different variances. Which is preferred?
Try an exam question
is a random sample from a population with mean and variance . The estimator is used to estimate . Show that is unbiased, find , and state with a reason whether or is the better estimator.
[4 marks]
- [1], so is unbiased.
- [1].
- [1] is smaller.
- [1]Both are unbiased, so is better because it has the smaller variance.
That's the notes covered.
Carry on to the next subtopic.
Try a question
Which statement about is correct?
1 mark, marked against the mark scheme
Which statement about is correct?
1 mark, marked against the mark scheme