This is a very old post, and google search took me here. For future references, it seems Adam is now adapted to do half-precision training with tuning on hyperparameters:

  
      programmersought.com
  
  
    

pytorch1.1 half-precision training Adam RMSprop optimizer Nan problem -... 171