It seems that you have a large model. Maybe recomputing in backward can help you save some memory. Here is a great PR to that end along with discussion and links to examples:

  
      github.com/pytorch/pytorch
  
  
      
    
  


  [checkpointing] Autograd container for trading compute for memory 77



  by prigoyal
  on 11:23PM - 10 Jan 18



  12 commits
  changed 3 files
  with 197 additions
  and 0 deletions.


  
  
    
    
  
  


Best regards
Thomas