Hi, I am training a simple LSTM for regression task and got following training loss curve.

Screenshot from 2017-10-18 21-05-42.png701×400 11.4 KB

While training, I logged the MSE computed on whole dataset at each epoch. The weird thing is that there is a flat region in the training curve. It seems that my network got stuck and learned nothing from epoch 2 to epoch 10. Then it got a ‘Aha moment’ at around epoch 11 and the loss dropped rapidly.
This really puzzled me. Any idea why this happened?
For code, please check here 12.