4.14 Linear combinations of random variables and unbiased estimates

4.14 Linear combinations of random variables and unbiased estimates

IB Maths: Applications and Interpretation HL: Revision notes

Studying this course? Keep your progress in one place.

Key facts

  • E(aX+b)=aE(X)+bE(aX+b)=aE(X)+b and Var(aX+b)=a2Var(X)Var(aX+b)=a^2Var(X).
  • For independent variables, variances add, even for X−YX-Y: Var(X−Y)=Var(X)+Var(Y)Var(X-Y)=Var(X)+Var(Y).
  • X1+X2X_1+X_2 (two independent copies) has variance 2Var(X)2Var(X), but 2X2X has variance 4Var(X)4Var(X).
  • The sample mean xˉ\bar x is an unbiased estimate of μ\mu.
  • The unbiased estimate of σ2\sigma^2 is sn−12=nn−1sn2s_{n-1}^2=\dfrac{n}{n-1}s_n^2.

One variable, linear transformation

The mean is shifted and scaled like each value; the variance is only scaled, by a2a^2.

If XX is a random variable and aa, bb are constants then E(aX+b)=aE(X)+bE(aX+b)=aE(X)+b.

For the variance, Var(aX+b)=a2Var(X)Var(aX+b)=a^2Var(X): adding bb does not change the spread and multiplying by aa multiplies the variance by a2a^2.

(You will not be asked to recall the variance formula for a single variable, but it explains the results for combinations.)

−4−22468100.10.20.30.4xyXY = 2X + 3
X ~ N(0, 1) and Y = 2X + 3 (illustrative): the mean shifts to 3 and the variance is multiplied by 2² = 4, so the curve is wider
  • Expected valueE(aX+b)=aE(X)+bE(aX+b)=aE(X)+b
  • VarianceVar(aX+b)=a2Var(X)Var(aX+b)=a^2Var(X)

Worked example

E(X)=120E(X)=120 and income I=15X+200I=15X+200. Find E(I)E(I).

Var(X)=4Var(X)=4. What is Var(3X+5)Var(3X+5)?

Linear combinations

Means always combine linearly; variances combine with squared coefficients when the variables are independent.

For any random variables, E(a1X1+⋯+anXn)=a1E(X1)+⋯+anE(Xn)E(a_1X_1+\cdots+a_nX_n)=a_1E(X_1)+\cdots+a_nE(X_n), whether or not they are independent.

If the variables are independent, Var(a1X1+⋯+anXn)=a12Var(X1)+⋯+an2Var(Xn)Var(a_1X_1+\cdots+a_nX_n)=a_1^2Var(X_1)+\cdots+a_n^2Var(X_n).

So for independent XX and YY: Var(X+Y)=Var(X)+Var(Y)Var(X+Y)=Var(X)+Var(Y) and, because (−1)2=1(-1)^2=1, Var(X−Y)=Var(X)+Var(Y)Var(X-Y)=Var(X)+Var(Y). Variances add even when the variables are subtracted.

24681012141618200.10.20.30.4xyXYX + Y
X ~ N(5, 1) and Y ~ N(8, 4), independent (illustrative): X + Y has mean 5 + 8 = 13 and variance 1 + 4 = 5
  • MeanE(aX±bY)=aE(X)±bE(Y)E(aX\pm bY)=aE(X)\pm bE(Y)
  • Variance (independent)Var(aX±bY)=a2Var(X)+b2Var(Y)Var(aX\pm bY)=a^2Var(X)+b^2Var(Y)

Worked example

E(X)=120E(X)=120, Var(X)=64Var(X)=64, E(Y)=90E(Y)=90, Var(Y)=36Var(Y)=36, with XX and YY independent. Find E(X+Y)E(X+Y) and Var(X−Y)Var(X-Y).

XX and YY are independent with Var(X)=9Var(X)=9 and Var(Y)=4Var(Y)=4. Find Var(X−Y)Var(X-Y).

Sum of copies versus doubling

Two independent copies partly cancel each other's extremes, but doubling one variable doubles every extreme.

If X1X_1 and X2X_2 are two independent copies of XX (for example two separate drivers) then E(X1+X2)=2E(X)E(X_1+X_2)=2E(X) and Var(X1+X2)=2Var(X)Var(X_1+X_2)=2Var(X).

For one variable doubled, E(2X)=2E(X)E(2X)=2E(X) but Var(2X)=4Var(X)Var(2X)=4Var(X).

The means agree but the variances differ: in X1+X2X_1+X_2 a high value of one can be offset by a low value of the other. Check whether the question describes separate independent items or one item counted several times.

0246810121416Var(X)Var(X₁ + X₂)Var(2X)QuantityVariance
Variances when Var(X) = 4 (illustrative): doubling one variable gives 4Var(X), two independent copies give 2Var(X)

X1 + X2

Meaning:
Two independent copies
Mean:
2E(X)
Variance:
2Var(X)

2X

Meaning:
One variable doubled
Mean:
2E(X)
Variance:
4Var(X)

Var(X)=5Var(X)=5. Find Var(X1+X2+X3)Var(X_1+X_2+X_3) for three independent copies, and Var(3X)Var(3X).

Unbiased estimate of the mean

An estimate is unbiased if its expected value equals the parameter: on average it is neither too high nor too low.

A sample is used to estimate the properties of a population. The sample mean is an unbiased estimate of the population mean:

xˉ=∑xin(or xˉ=∑fixin for a frequency table),\bar x=\frac{\sum x_i}{n}\quad(\text{or }\bar x=\frac{\sum f_ix_i}{n}\text{ for a frequency table}),

so E(Xˉ)=μE(\bar X)=\mu. (The proof is not examined.)

468101214160.20.40.60.81xydistribution of x̄
Distribution of the sample mean for a population with μ = 10, σ = 2 (illustrative): it is centred on μ for every sample size n, so x̄ is unbiased
  • Sample meanxˉ=∑xin\bar x=\dfrac{\sum x_i}{n}
  • UnbiasedE(Xˉ)=μE(\bar X)=\mu

Worked example

Masses of six eggs: 58,61,63,60,57,6558, 61, 63, 60, 57, 65 g. Find an unbiased estimate of μ\mu.

What does "unbiased" mean for an estimate?

Unbiased estimate of the variance

Divide by n−1n-1, not nn, to get an unbiased estimate of the population variance.

The sample variance dividing by nn,

sn2=∑fi(xi−xˉ)2n,s_n^2=\frac{\sum f_i(x_i-\bar x)^2}{n},

systematically underestimates σ2\sigma^2. The unbiased estimate of variance divides by n−1n-1:

sn−12=nn−1sn2=∑fi(xi−xˉ)2n−1.s_{n-1}^2=\frac{n}{n-1}s_n^2=\frac{\sum f_i(x_i-\bar x)^2}{n-1}.

On most GDCs, σx\sigma_x is sns_n and sxs_x is sn−1s_{n-1} (check your model). Square it to get the variance.

01234Divide by nDivide by n − 1DivisorVariance estimate
Data 2, 4, 6 (mean 4, sum of squared deviations 8): dividing by n underestimates the variance
  • Unbiased variancesn−12=nn−1sn2s_{n-1}^2=\dfrac{n}{n-1}s_n^2

Worked example

For the six eggs sn2=7.56s_n^2=7.56. Find the unbiased estimate of σ2\sigma^2.

n=10n=10 and sn2=9s_n^2=9. Find sn−12s_{n-1}^2.

Using estimates in a model

Treat xˉ\bar x and sn−12s_{n-1}^2 as μ\mu and σ2\sigma^2, and say they are estimates.

Once xˉ\bar x and sn−12s_{n-1}^2 are found, use them as μ\mu and σ2\sigma^2 in the linear combination formulas.

State clearly that these are estimates. Keep unrounded values until the final answer and give 33 significant figures (money to 22 decimal places).

  1. 1

    Estimate

    find the sample mean and s(n-1) squared

  2. 2

    Substitute

    use them as mu and sigma squared

  3. 3

    Combine

    apply the E and Var rules

  4. 4

    Round last

    3 s.f., money 2 d.p.

From sample to model

Worked example

Parcels per driver: xˉ=10.375\bar x=10.375 and sn−12=1.676s_{n-1}^2=1.676. Payment is P=6X+40P=6X+40 AED. Find E(P)E(P), and E(X1+X2)E(X_1+X_2) and Var(X1+X2)Var(X_1+X_2) for two independent drivers.

When using xˉ\bar x and sn−12s_{n-1}^2 in a combination formula, what do they stand for?

Try an exam question

XX and YY are independent random variables with E(X)=50E(X)=50, Var(X)=16Var(X)=16, E(Y)=30E(Y)=30 and Var(Y)=9Var(Y)=9. Let W=2X−YW=2X-Y. Find E(W)E(W) and Var(W)Var(W).

[4 marks]

That's the notes covered.

Carry on to the next subtopic.

Try a question

Find E(X+Y)E(X+Y).

1 mark, marked against the mark scheme

Revision notes