@ptrblck Thank you for your reply.
Below are details about configuration.
PyTorch --> 1.4.0
CUDA --> 10.2
I am not able to locate CuDNN. If CuDNN comes inbuilt with the CUDA, then it might be installed, but I haven’t installed it myself.
Yes, I have checked on both GPU and CPU I am getting NaN just after 5 to 10 iterations (i.e. 5 to 10 batches).