I think my new question should be expressed like this: Since hidden state and cell state are not learned parameters, then why do we have to clear gradients and init hidden states before we train the next batch or next example?  What does the following sentence mean? Im sorry cause I used to use TF or just coding with numpy so im not so familiar with pytorch, Thanks again!



 emon7fhz:

Pytorch accumulates gradients.