@joeddav @patrickvonplaten I spent my hard earned internet points on SO to find the problem:

  
      stackoverflow.com
  
  
      
    
  

  Gradients returning None in huggingface module 22



  python, nlp, pytorch, huggingface-transformers



  
  answered by
  
    Roger Trullo
  
  on 11:03PM - 16 Nov 20 UTC


  
  
    
    
  
  


It turns out that calling the transpose:
encoder_states = tuple(hidden_state.transpose(0, 1) for hidden_state in encoder_states)

kills the backprop for some reason. I’m not sure why this is, but it solved my problem. It might be nice to not have to monkey patch this to get the gradients.