@jpeg729  I am trying to train a model the way you are suggesting and I have a couple questions about it.
1- If i call loss.backward() after every batch wouldnt this reset the graph and therefore detach the hidden_state so that it won’t be used in the next batch?
2- At the beginning of each batch i reset the LSTM hidden state and cell state to 0. Is this a valid approach if I want to exclusively remember one batch?
3- If i want to retain the last hidden state from the previous batch and avoid backpropagating through the beginning. Couldn’t I just call loss.backward() at the end of one batch and leave the hidden state untouched?
I would love some clarification regarding this. Thanks in advance!
Thanks in advance!