Everyone imports LinearRegression from scikit-learn and moves on. Today we build it by hand — and you'll see that "training a model" is just a line sliding into a cloud of points.

This is Day 1 of MachineLearningFromZero: every core algorithm built from scratch, no black-box library.

The model is two numbers

let m = Math.random(), b = Math.random();   // slope + intercept
const predict = x => m * x + b;              // ŷ = m·x + b

That's the whole "model". Training means finding the m and b that fit the data.

The loss: how wrong are we?

For each point, take (prediction − actual), square it (so big misses hurt more and signs don't cancel), and average. That's Mean Squared Error:

const mse = () =>
  data.reduce((s, [x, y]) => s + (predict(x) - y) ** 2, 0) / data.length;

Lower MSE = better line. Training = making this number small.

Gradient descent: roll downhill

The gradient tells us which way the loss increases. We want it to decrease, so we step the opposite way. Calculus gives clean formulas for the slope of MSE with respect to m and b:

function step(lr) {
  let dm = 0, db = 0;
  for (const [x, y] of data) {
    const err = predict(x) - y;   // how wrong, and which direction
    dm += 2 * err * x;            // ∂loss/∂m
    db += 2 * err;                // ∂loss/∂b
  }
  dm /= data.length; db /= data.length;
  m -= lr * dm;                   // step downhill
  b -= lr * db;
}

lr is the learning rate — the step size. Too small and training crawls; too big and it overshoots and the loss explodes. This one knob is the most important hyperparameter in all of ML.

Train = repeat the step

for (let epoch = 0; epoch < 200; epoch++) step(0.1);

As the line nears the best fit, the gradients shrink toward zero, so the steps shrink too — it settles on its own. Then prediction is trivial:

const score = predict(4.5);   // m * 4.5 + b

Why this matters

This exact loop — model → loss → gradient → step — is the engine under logistic regression, neural networks, and deep learning. Master it here on a single line and every later algorithm is a variation on this theme.

🌐 Watch the line learn live (drag the learning rate, hit Train): https://dev48v.infy.uk/ml/day1-linear-regression.html

Day 1 of MachineLearningFromZero. A new algorithm every day, built from scratch.