Solved by ptrblck in post #2 
              
              
                Calling model.half() might work, but can easily create overflows and thus NaNs. 
We recommend to use the automatic mixed-precision training, which takes care of these issues and also stabilizes the training using loss scaling. 
You can use it in the nightly binaries or by building PyTorch from sourc…