4.13 Non-linear regression

4.13 Non-linear regression

IB Maths: Applications and Interpretation HL: Revision notes

Studying this course? Keep your progress in one place.

Key facts

  • Least squares regression chooses the model with the smallest sum of squared residuals.
  • Exam models: linear, quadratic, cubic, exponential y=abxy=ab^x, power y=axby=ax^b and sine; use your GDC.
  • A smaller sum of squared residuals means a closer fit to the same data.
  • R2R^2 is the proportion of the variability in yy accounted for by the model; for a linear model R2=r2R^2=r^2.
  • R2R^2 alone is not enough to judge a model: think about context, range and complexity.

Fitting curves by least squares

The least squares curve is the one that makes the sum of the squared vertical distances to the data as small as possible.

In regression we fit a model so that the vertical distances between the points and the curve are small. The least squares regression curve minimises the sum of the squares of these distances.

The distance between an observed value yiy_i and the model's value y^i\hat y_i is the residual.

In examinations you use technology: enter the data in lists, choose the regression curve type and read off the parameters (to 33 significant figures unless told otherwise). Always state the type and write the full equation.

residualD1D2D3D4D5D6D7
Illustrative: the least squares curve minimises the sum of the squares of the residuals, the vertical distances from data to curve
  • Least squaressmallest ∑(yi−y^i)2\sum(y_i-\hat y_i)^2
  • Residualyi−y^iy_i-\hat y_i

What does least squares regression minimise?

The models and their forms

Choose the model from the shape of the data and the context.

  • Quadratic: y=ax2+bx+cy=ax^2+bx+c. Cubic: y=ax3+bx2+cx+dy=ax^3+bx^2+cx+d.
  • Exponential: y=abxy=ab^x (or aekxae^{kx}). At equal steps the yy values form a geometric sequence; b=1.38b=1.38 means 38%38\% growth per unit.
  • Power: y=axby=ax^b.
  • Sine: y=asin⁡(bx+c)+dy=a\sin(bx+c)+d: amplitude ∣a∣|a|, period 2πb\frac{2\pi}{b}, principal axis y=dy=d.
123456510152025xy(0, 3.5)(6, 24.2)N = 3.5 × 1.38^t
Exponential model N = 3.5 × 1.38^t with its start and end values.

Which model has the variable in the exponent?

Sum of squared residuals

A smaller sum of squared residuals means a closer fit to the same data.

The sum of squared residuals, SSres=∑(yi−y^i)2SS_{res}=\sum(y_i-\hat y_i)^2, measures how closely the model fits the data.

It depends on the units and size of the data, so use it to compare models fitted to the same data. The least squares curve has the smallest SSresSS_{res} of its type.

For braking-distance data, SSres=79.0SS_{res}=79.0 for the linear model and 4.674.67 for the quadratic, so the quadratic fits much more closely.

020406080LinearQuadraticModelSum of squared residuals
Sum of squared residuals for two models of the same braking-distance data (smaller is better).

Model A has SSres=12.5SS_{res}=12.5 and model B has SSres=3.2SS_{res}=3.2 on the same data. Which fits better?

The coefficient of determination

R2R^2 is the proportion of the variability in yy accounted for by the model.

The coefficient of determination R2R^2 is found using technology. It lies between 00 and 11; closer to 11 means a better fit.

For example R2=0.998R^2=0.998 means 99.8%99.8\% of the variation in braking distance is accounted for by the model.

For a linear model, R2=r2R^2=r^2, where rr is Pearson's product-moment correlation coefficient. If r=0.982r=0.982 then R2=0.964R^2=0.964.

02040608010012030507090110Speed / km h⁻¹Braking distance / m
  • Data
  • Quadratic model
Illustrative braking distances: model and data almost coincide, so R² is close to 1
  • Linear modelR2=r2R^2=r^2
  • R2R^2 close to 1better fit

A linear model has r=−0.9r=-0.9. What is R2R^2?

Why R² is not enough

Many factors affect validity, so R2R^2 alone should not decide between models.

  • Context: does the model make sense? A linear braking model gives −15.9-15.9 m at 00 km/h, which is impossible.
  • Range: using a model outside the data range is extrapolation, less reliable than interpolation.
  • Complexity: a cubic often has a higher R2R^2 than a quadratic without being better.
  • Sample size: small samples can fit by chance.

Give a reason that refers to the context or the data, not just "R2R^2 is high".

020406080100120030507090110Speed / km h⁻¹Braking distance / m
  • Data
  • Linear model
  • Quadratic model
Illustrative: the linear model predicts −15.9 m at 0 km/h, which is impossible, so a good fit alone does not make it valid

Interpolation

  • Inside the data range
  • More reliable

Extrapolation

  • Outside the data range
  • Trend may not continue

A model is used to predict a value far outside the data range. What is this called and why is it risky?

Choosing and using a model

Pick the model from the data's shape, fit it on the GDC, then use and interpret it.

Culture data: t=0t=0 to 66 hours, N=3.5,4.8,6.7,9.2,12.7,17.5,24.2N=3.5,4.8,6.7,9.2,12.7,17.5,24.2 thousand. Successive ratios are about 1.381.38, so an exponential model suits.

For a periodic tide use sine regression. With h=1.30sin⁡(0.508t−1.02)+2.60h=1.30\sin(0.508t-1.02)+2.60 the amplitude is 1.301.30, the period is 2π0.508=12.4\frac{2\pi}{0.508}=12.4 hours and the mean depth is 2.602.60.

246810121234xymean 2.60max 3.90min 1.30h = 1.30 sin(0.508t − 1.02) + 2.60
Tide model: amplitude 1.30, mean depth 2.60.

Worked example

Use the culture data above to model NN and predict NN at t=8t=8.

For y=asin⁡(bx+c)+dy=a\sin(bx+c)+d with b=0.5b=0.5 and xx in radians, what is the period?

Try an exam question

A linear model for data has Pearson's correlation coefficient r=−0.9r=-0.9. A quadratic model for the same data has a smaller sum of squared residuals. (a) Find R2R^2 for the linear model and interpret it. (b) Say which model fits better. (c) Give one reason why R2R^2 alone is not enough to choose a model.

[4 marks]

That's the notes covered.

Carry on to the next subtopic.

Try a question

Using exponential regression on your GDC, which model fits the data?

1 mark, marked against the mark scheme

Revision notes