Hi
This is not fixed as far as I see, could you have a look here please:

  
      github.com/huggingface/transformers
  
  
    
  
	  
  

  
    
      getting nans with t5-large + fix 68
    

    
      
        opened 
        
          
        
        Mar 21, 2021
      
      


      
        
          
          yuvalkirstain
        
      
    
  



  Environment info
transformers version: 4.5.0.dev0
Platform: Linux-4.15.0-65-generic-x86_64-with-glibc2.10
Python version: 3.8.8
PyTorch version (GPU?): 1.7.1+cu101 (True)
Tensorflow version (GPU?): not installed (NA)
Using GPU in script?: Yes
Using distributed...





  
  
    
    
  
  


  
      github.com/huggingface/transformers
  
  
    
  
	  
  

  
    
      mt5 getting nans with fp16 48
    

    
      
        opened 
        
          
        
        Mar 20, 2021
      
      


      
        
          
          dorost1234
        
      
    
  



  Environment info

transformers version: 4.4.2
Platform: linux
Python version: 3.7
PyTorch version (GPU?): 1.8
Tensorflow version (GPU?): -
Using GPU in script?: -
Using distributed or parallel set-up...





  
  
    
    
  
  


I really need these models and I appreciate having a look into it.
thanks a lot