Solved by ptrblck in post #5 
              
              
                Checking each value for an overflow could yield bad general performance. 
To debug an invalid value, you could use torch.autograd.detect_anomaly.