Hi,
You are using MSE then the only reason that can cause this is either your model definition is wrong or your targets are not correct numbers.
The issue is not related to learning rate or other hyperparamters as your first iteration is also producing nan. Simply test your network with a random input and see if it’s producing a proper output and also verify your targets.
Bests