Specialty · AILesson 2 of 6Free to read

Linear regression and loss · AI Foundations

How do you find the best straight line through noisy points?

Why you need it

Linear regression is the simplest model that learns: predict a number from a feature with a straight line. It's still used everywhere for quick estimates, and it introduces the two ideas every model shares: a prediction function with adjustable numbers, and a loss that scores how wrong the predictions are.

The intuition

The model is a line with two numbers you can tune, the weight w (slope) and the bias b (where it crosses zero):

prediction = w × x + b

To compare lines, measure how far each prediction is from the real value, square it, and average. That's the mean squared error (MSE):

MSE = average of (prediction − actual)²

Squaring does two things: errors above and below can't cancel out, and a big miss counts much more than a small one. The best line is the one with the lowest MSE.

For one feature there's an exact formula (least squares). With x̄ and ȳ the averages:

w = Σ (x − x̄)(y − ȳ) / Σ (x − x̄)²
b = ȳ − w × x̄

In the next lesson you'll find the same line a different way, step by step, which also works for models with no formula.

Definition

Linear regression

Predicting a number with a straight line: w × x + b.

Weight and bias

The tunable numbers of a model. In a line, the weight is the slope and the bias is the value at x = 0.

Loss

A number that says how wrong the predictions are. Training tries to make it small.

Mean squared error

The average of the squared differences between predictions and actual values.

Examples

Score two lines

Try it yourself

The line that fits better has the lower loss.

xs = [1, 2, 3, 4]
ys = [3, 5, 7, 9]

def predict(w, b, x):
    return w * x + b

def mse(w, b):
    errors = [(predict(w, b, x) - y) ** 2 for x, y in zip(xs, ys)]
    return sum(errors) / len(errors)

print(mse(1, 1))
print(mse(2, 1))

Output

7.5
0.0

Step by step

L8One squared error per point.
L11Predictions 2, 3, 4, 5 miss by 1, 2, 3, 4: (1 + 4 + 9 + 16) / 4 = 7.5.7.5
L12y = 2x + 1 goes through every point: no error at all.0.0

The least-squares line

Try it yourself

Real data is noisy; the formula still finds the best line.

xs = [1, 2, 3, 4, 5]
ys = [2.1, 3.9, 6.2, 7.8, 10.1]

mx = sum(xs) / len(xs)
my = sum(ys) / len(ys)
w = sum((x - mx) * (y - my) for x, y in zip(xs, ys)) / sum((x - mx) ** 2 for x in xs)
b = my - w * mx
print(round(w, 2), round(b, 2))
print(round(w * 6 + b, 1))

Output

1.99 0.05
12.0

Step by step

L4Averages: x̄ = 3, ȳ = 6.02.
L6How x and y move together, divided by how much x moves.
L7The best line always passes through the point of averages.
L9Predict x = 6 with the learned line.

Common mistakes

A line through a curve

If the points bend, the best straight line is still wrong in a systematic way. Plot the data first.

Predicting far outside the data

A line learned from flats of 40–120 m² says nothing reliable about a 900 m² mansion.

One outlier pulls the line

Squared error punishes big misses heavily, so one wrong row (a typo with an extra zero) can tilt the whole line. Clean the data.

Quick quiz

1. What does this print?

actual = [10, 20, 30, 40]
predicted = [12, 18, 30, 44]
errors = [(p - a) ** 2 for p, a in zip(predicted, actual)]
print(sum(errors) / len(errors))

A
```
6.0
```
B
```
2.0
```
C
```
24
```

Show the answer

Answer: A

Squared errors 4, 4, 0, 16 sum to 24; the mean is 6.0.

2. Why square the errors instead of just averaging them?

ASo errors above and below can't cancel out, and big misses count more
BBecause negative numbers can't be averaged
CTo make the loss easier to read

Show the answer

Answer: A

+5 and −5 would average to 0 and hide two bad predictions.

Practice

3 exercises in three steps: predict what a program prints, change a program to meet a new requirement, then write one from scratch. Your code runs in the browser and is checked the moment you submit.

PredictWalk along a line
Type exactly what this prints.
ModifyErrors that cancel
This loss reports a perfect 0.0 for predictions that are clearly wrong: one too high and one too low cancel out. Fix mse so it returns the mean squared error.
WriteFit the line
Write fit(xs, ys) that returns (w, b) for the least-squares line, using the formula from this lesson. xs always has at least two different values.
```
fit([1, 2, 3, 4], [3, 5, 7, 9])   # (2.0, 1.0)
```

Practise this lesson

Open the interactive lesson to edit and run these examples and solve the exercises with instant feedback. Free to start, no sign-up.

Open the interactive lesson Course map

Interactive lessons open in order. If you have not done the earlier ones yet, you start at the course map.

Terms in this lesson

Term	Chinese	Definition
Linear regression	线性回归	Predicting a number with a straight line.
Loss	损失	A score for how wrong predictions are.
Mean squared error	均方误差	The average of squared prediction errors.