Could you give us some information about your setup?
I.e. which PyTorch, CUDA, cudnn versions are you using?
Are you seeing these NaNs on the CPU, GPU or both?
Are you able to reproduce the NaNs quickly (in a couple of iterations)?