Calling model.half() might work, but can easily create overflows and thus NaNs.
We recommend to use the automatic mixed-precision training 72, which takes care of these issues and also stabilizes the training using loss scaling.
You can use it in the nightly binaries or by building PyTorch from source.