Hi,
If the values become bigger and bigger, this is the issue. The fact that it ends up so big that you get nan is not the root problem.
You just want to make sure your network stops diverging.
Do you properly zero out the gradients between iterations? Do you properly set the learning rate and other optimizer settings?