Hi @Atcold,
Why do you think the hidden state needs to be forwarded ?
Correct me if I’m wrong but, isn’t the whole purpose of using nn.LSTM over nn.LSTMCell is not having to iterate over the hidden state for each time step (of the sequence)? (Making hidden state kind of irrelevant outside LSTM) ? 
cc: @apaszke