Make one weight learn
Take real gradient steps in a tiny classifier and track the loss.
TRY TO FIND OUT
What does changing the learning rate change?
WHAT YOU ARE LOOKING AT
A trained scalar logistic model; much smaller than a language model. The result is an educational calculation, not a claim about a commercial model’s internal state.
CONNECT THE IDEA
Gradient descent: changing the weights
A gradient points uphill in loss. An optimizer uses it to take a carefully sized step in the other direction.