This is quite surprising.
But if the loss doesn’t diverge, you want to add prints in your evaluation code to see exactly where the first nan appears. either because of a division by 0 or something similar. and make sure it doesn’t happen anymore (adding an eps to division)