@jpeg729 there is something basic I’m missing here regarding the 2nd forward pass you say is not needed. I verified it in toy examples and you are correct by removing it.
the reason I think it’s more accurate to do the second forward pass is that you want the next batch to get a hidden state that reflects the weights update.
meaning, after the weights are updated, rerunning the batch with the updated weights will result a different hidden state. other wise we are feeding the next batch with a hidden state that was generated by the forward pass before the weight update.
can anyone explain to me what am I missing here?