How learning happens
Connect data, prediction error, gradients, and updates. Make one small weight learn before considering a large training run.
By the endDistinguish a forward pass, a backward pass, and an optimizer update.
Start path: Training data: what the model gets to learn from →6 lessons · 36 min total
Mark lessons as read to save your place on this browser.
- 16 MIN READING · Not read yet
Training data: what the model gets to learn from
Identify the material that supplies training examples.
Read the lesson → - 26 MIN READING · Not read yet
Data quality: more than removing typos
Consider what filtering changes and misses.
Read the lesson → - 36 MIN READING · Not read yet
Loss: a number to learn from
Measure how much probability the target received.
Read the lesson → - 46 MIN READING · Not read yet
Backpropagation: tracing responsibility
Trace how parameters contributed to the loss.
Read the lesson → - 56 MIN READING · Not read yet
Gradient descent: changing the weights
Use gradients to update parameters.
Read the lesson → - 66 MIN READING · Not read yet
Pretraining: learning from continuations
Repeat across batches while checking held-out data.
Read the lesson →