This is for doing truncated BPTT. The dependency graph of RNN can be simply viewed as this.
c1, h1 -> c2, h2 -> c3, h3 -> ... -> cn, hn
    |        |         |                |
  loss1    loss2     loss3            lossn

If we did not truncate the history of hidden states (c, h), the back-propagated gradients would flow from the loss towards the beginning, which may result in gradient vanishing or exploding. Detailed explanation can be found here.

  
      WildML – 8 Oct 15
  
  
    

Recurrent Neural Networks Tutorial, Part 3 – Backpropagation Through Time and... 729

This the third part of the Recurrent Neural Network Tutorial. In the previous part of the tutorial we implemented a RNN from scratch, but didn’t go into detail on how Backpropagation Through …