I was unaware that you could call LSTM.forward without providing the hidden states.
According to the source code 78, if you don’t provide the hidden states, then it creates new blank ones and doesn’t store them in between batches.
Just so we are clear, in the example I gave, the variable hidden contains both the hidden and cell states.