I think this repo is dedicated to explore mixed-precision with PyTorch.

  
      
      GitHub
  
  
    

NVIDIA/apex 44

A PyTorch Extension:  Tools for easy mixed precision and distributed training in Pytorch - NVIDIA/apex


  
  
    
    
  
  


I’m running the scripts with the World Language Model, and can see a slight performance boost (+15%) with --fp16 on the 2070.