maybe the this discussion might help:

  
    
    
    RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time 
  
  
    Thanks for your quick reply! 
Actually I’m trying to implementing an attention mechanism part. So I need these hidden state to multiply the attention weight.