Hi all,
I would agree that it is very likely due to the gradient explosion.
https://machinelearningmastery.com/exploding-gradients-in-neural-networks/ 1.2k
This URL is very helpful.
Besides clipping, this URL also suggest using ReLU, LSTM, etc…
Wish it helps!
Yisong