Mate,  You’d  better   post   your  network  type  here  , and   what   task  you   are     training. (Classfication  or   object  detection)
Rely :
I   often  encouter   this  problem  in object detection,  when  I   use    torch.log(a),if  a  is   negative  number . It  will  be  nan  ,  because   your  loss    function  will  get  a    nan  value  as   parameter .
So  you   should   debug   your  code   step   by   step  , and  see   which step   the  tensor   suddenly  becomes   nan