@aksve This has been explained in detail in the below post

  
    
    
    [solved] Why we need to detach Variable which contains hidden representation? 
  
  
    I was going through the pytorch official example - “word_language_model” and found the following line of code in the train() function. 
# Starting each batch, we detach the hidden state from how it was previously produced.
# If we didn't, the model would try backpropagating all the way to start of the dataset.
hidden = repackage_hidden(hidden)

I am not understanding why we need to detach hidden variable from the hidden variable associated with the previous batch of input? When gradients are com…