Hello,
I have a quite odd problem with my loss function during training. I have a function train(n_epochs, batch_size) that performs training with a certain number of epochs and a certain batch_size. The problem is that when i restart training, the loss suddenly increases.
For example, in the following figure we can see that the loss suddenly increases around 30000 iterations, that corresponds to a second training I started. At first, I thought this was due to the running means on the batch normalization, but I reset them and that does not seem to change anything.

image.png651×535 24.2 KB

Thanks in advance,
Gabriel