Hi @ptrblck,
Thanks for the reply.
The task I am working on is similar to Super-resolution, and the loss we use is MSE.
so basically we mesure the difference between each pixel and use MSE to calculate the loss.
for 1 epoch, there are 20000 iterations, and loss in 1st epoch will decrease from 10M+ to 6M+,
but instantly drop to around 10 at the begining of 2nd epoch.
for random input the loss is very high, just like what I observed during training.
Thus I still dont understand why second epoch the loss will drop that much…though the model gives good test result in the end anyway…