OK, something seems to be broken. Could you post the whole code?
As far as I can tell, the current code looks good.
If you cannot post the code due to your work policy, could you have a look at the norm of the gradients in the first and second epoch?