\sum xy-\dfrac{\sum x\sum y}{n}

What does the least squares method minimise?

The sum of the squares of the vertical distances (residuals) from the points to the line.

Observed y minus predicted \hat y: yi-(a+bxi).

What does a negative residual mean?

The point lies below the regression line (the model overestimates).

Formula for the residual sum of squares?

RSS=S{yy}-\dfrac{(S{xy})^2}{S{xx}}

What does a pattern in the residuals suggest?

A linear model may not be suitable (for example, curvature).

What is extrapolation, and why is it risky?

Predicting outside the data range; the linear relationship may not continue.

How do you decide whether to remove an outlier?

Only if there is a reason (such as a recording error); then recalculate the line and RSS and compare.

Least squares regression and residualsEdexcel A-Level Further Maths: Flashcards

Card 1 of 130 of 13 known

Question

Formula for the gradient $b$ of the regression line of $y$ on $x$?

Tap or press Space to reveal

Tap card or press Space to flip

See all 13 cards

Formula for the gradient $b$ of the regression line of $y$ on $x$ ?: $b=\dfrac{S_{xy}}{S_{xx}}$
Formula for the intercept $a$ ?: $a=\bar y-b\bar x$
Formula for $S_{xy}$ ?: $\sum xy-\dfrac{\sum x\sum y}{n}$
Formula for $S_{xx}$ ?: $\sum x^2-\dfrac{(\sum x)^2}{n}$
What does the least squares method minimise?: The sum of the squares of the vertical distances (residuals) from the points to the line.
Define a residual.: Observed $y$ minus predicted $\hat y$ : $y_i-(a+bx_i)$ .
What does a negative residual mean?: The point lies below the regression line (the model overestimates).
Formula for the residual sum of squares?: $\text{RSS}=S_{yy}-\dfrac{(S_{xy})^2}{S_{xx}}$
What do the residuals of a least squares line sum to?: Zero.
Which point does the regression line always pass through?: $(\bar x,\bar y)$
What does a pattern in the residuals suggest?: A linear model may not be suitable (for example, curvature).
What is extrapolation, and why is it risky?: Predicting outside the data range; the linear relationship may not continue.
How do you decide whether to remove an outlier?: Only if there is a reason (such as a recording error); then recalculate the line and RSS and compare.

Exam questions on Least squares regression and residuals

A researcher models the relationship between a variable $x$ and a variable $y$ using the least squares regression line of $y$ on $x$ , $y=a+bx$ . Summary statistics from the data give $\bar x=5$ , $\bar y=14$ , $S_{xx}=40$ and $S_{xy}=88$ .
One observation is $x=6$ , $y=15$ . Calculate the residual for this observation.2 marks
A student records the number of hours, $x$ , spent revising and the score, $y$ , out of 40, on a test for 5 students: $(1,12)$ , $(2,15)$ , $(3,21)$ , $(4,22)$ , $(5,30)$ . The summary statistics are $n=5$ , $\sum x=15$ , $\sum y=100$ , $\sum x^2=55$ , $\sum xy=343$ and $\sum y^2=2194$ .
The regression line is $y=7.1+4.3x$ . Interpret the value $4.3$ in context, and explain why the line should not be used to predict the score of a student who revises for 15 hours.2 marks
A café owner records the midday temperature, $x\,^\circ$ C, and the number of cold drinks sold, $y$ , on six days: $(14,40)$ , $(16,45)$ , $(18,55)$ , $(20,58)$ , $(22,68)$ , $(24,76)$ . The summary statistics are $n=6$ , $\sum x=114$ , $\sum y=342$ , $\sum x^2=2236$ , $\sum xy=6750$ and $\sum y^2=20414$ .
Find the equation of the regression line of $y$ on $x$ .3 marks

See the full worksheet

Written by the Exaim team, led by Shaun Daswani (Head of Upper Secondary, Improve ME Institute; MSc Financial Mathematics, Imperial College London; BSc, UCL) and Jason Daswani (operational lead, Improve ME Institute; LSE).