Archived May, 2026.

Computing a regression line

mark

Consider the data

\{(0,0),(1,0), (2,1)\}.

We'd like to fit a model of the form f(x)=ax+b to that data.

  • Write down the the total squared error E(a,b) for that data and model,
  • Compute the derivatives \frac{\partial E}{\partial a} and \frac{\partial E}{\partial b}.
  • Set those partial derivatives equal to zero and solve the resulting system to obtain the regression line, and
  • Plot the data together with the regression line.
User 010

When given the data {(0,0),(1,0),(2,1)}, we can do the following.

The total squared error E(a,b) for the data and model is

E(a,b) = b^2 + (a+b)^2 + (1-2a-b)^2.

We then compute the partial derivatives for a and b respectively.

\frac{\partial E}{\partial a} = 10a + 6b - 4
\frac{\partial E}{\partial b} = 6a + 6b - 2

I then set both equations equal to zero.

10a + 6b - 4 = 0
6a + 6b - 2 = 0

I subtracted the bottom equation from the top to cancel out the b's and got:

4a = 2

Which means:

a = 1/2

I then plugged a back into one of the equations, I chose the second:

6(1/2) + 6b = 2
3 + 6b = 2
6b = -1
b = -1/6

Finally, I utilized both a and b in the form f(x) = ax + b, plugging them both in to get:

f(x) = (1/2)x - 1/6

The plotted data with the regression line: