PabloRR100:

What I am missing?


In the first example, that’s the average loss for a minibatch (in that example, ~2000 training examples). The reason why it’s restarted is because it’s used to track progress during training in terms of: does the loss go up or down with the next minibatches.
Usually, the loss is magnitudes higher during the first minibatches (or even epochs) when you start the training, and you would likely lose the signal in all that noise if you would keep a running average over all training examples over all epochs. I.e., the longer you keep training, the harder it will be to detect what’s currently going on in your epoch because of the averaging effect.