In the document, the examples using LSTM look like:
>>> rnn = nn.LSTM(10, 20, 2)
>>> input = torch.randn(5, 3, 10)
>>> h0 = torch.randn(2, 3, 20)
>>> c0 = torch.randn(2, 3, 20)
>>> output, (hn, cn) = rnn(input, (h0, c0))

Where (hn, cn) contains the hidden state and cell state of current LSTM. So you can use (hn, cn) outputed from the previous timestep as the input of LSTM at the next timestep.