Type I and Type II errors
Studying this course? Keep your progress in one place.
Key facts
- A Type I error is rejecting when it is true; a Type II error is not rejecting it when it is false.
- Describe each error in context: what is concluded and what is really true.
- calculated with the parameter from .
- A larger critical region raises the probability of a Type I error.
- At least one false rejection in tests: .
Type I and Type II errors
A test can reject a true (Type I) or fail to reject a false one (Type II).
A hypothesis test is a decision based on sample evidence, so it can be wrong.
- A Type I error is rejecting when it is true.
- A Type II error is failing to reject when it is false.
You must be able to define each in the context of the question and calculate the probability of a Type I error.
| H0 is actually true | H0 is actually false | |
|---|---|---|
| Reject H0 | Type I error | Correct decision |
| Do not reject H0 | Correct decision | Type II error |
H0 is actually true
- Reject H0:
- Type I error
- Do not reject H0:
- Correct decision
H0 is actually false
- Reject H0:
- Correct decision
- Do not reject H0:
- Type II error
A test rejects H0 although H0 is true. What type of error is this?
Describing errors in context
Say what the test would conclude and what is actually true, in the words of the situation.
Write what the test would conclude and what is actually true, using the words of the situation, not just "" and "".
A seed company claims and a gardener tests .
- Type I: concluding that the probability of germination is below when it is in fact .
- Type II: concluding there is no evidence the probability is below when it is in fact lower.
| Type I error | Type II error | |
|---|---|---|
| Test concludes | Germination probability is below 0.8 | No evidence it is below 0.8 |
| Reality | It is actually 0.8 | It is actually lower |
Type I error
- Test concludes:
- Germination probability is below 0.8
- Reality:
- It is actually 0.8
Type II error
- Test concludes:
- No evidence it is below 0.8
- Reality:
- It is actually lower
A factory claims 2% of bolts are faulty () and a customer tests . Describe a Type II error.
Probability of a Type I error
Find the probability of landing in the critical region when is true, using the parameter from .
The probability of a Type I error is the probability that the test statistic falls in the critical region when is true. It is also the actual significance level.
Always use the value of the parameter given by . The method is the same for binomial and Poisson tests.
For a two-tailed critical region, add both tails.
- P(Type I error)
Worked example
against with and critical region . Find the probability of a Type I error.
- 1
Use , so .
- 2
.
- 3
.
Worked example
, , reject if . Find the probability of a Type I error.
- 1
Use .
- 2
.
- 3
.
, critical region for . Which expression gives the probability of a Type I error?
Choosing a critical region
Take the largest region whose probability is at most the significance level.
To test at the 5% level, take the largest critical region whose probability is at most .
Making the critical region larger increases the probability of a Type I error.
The actual significance level can be well below the nominal significance level of 5% because the distribution is discrete.
- 1
Try a boundary
find its probability under H0
- 2
Compare
is it at most the significance level?
- 3
Adjust
if too large, move one value inwards
- 4
State
the region and its actual significance level
Worked example
and . Find the critical region at the 5% level and the actual significance level.
- 1
Try : , so this is too large.
- 2
Try : , so this is allowed.
- 3
The critical region is .
The critical region is made larger. What happens to the probability of a Type I error?
Repeated testing
Each test has probability of a false rejection, so the chance of at least one is .
If a true is tested several times independently, each test has probability of a false rejection.
The probability of at least one wrong rejection in tests is
With and this is . Many tests make at least one false alarm quite likely.
Each test has . What is the probability of at least one Type I error in 2 independent tests of a true ?
Try an exam question
A test of against uses and rejects if . Find the probability of a Type I error and state what a Type I error means in this test.
[3 marks]
- [1]Under , and .
- [1]A Type I error is rejecting when it is true.
- [1]That is, concluding when in fact .
That's the notes covered.
Carry on to the next subtopic.
Try a question
Which of the following describes a Type I error?
1 mark, marked against the mark scheme
Which of the following describes a Type I error?
1 mark, marked against the mark scheme