The LSTM module returns two things each time you feed it data. 1. The output. 2. The hidden state. Like this…
# initialise LSTM layer
lstm = nn.LSTM(3, 3)
# initialise inputs and hidden states
inputs = torch.cat(inputs).view(len(inputs), 1, -1)
hidden = (autograd.Variable(torch.randn(1, 1, 3)), autograd.Variable(
    torch.randn((1, 1, 3))))
# feed the LSTM some data
out, hidden = lstm(inputs, hidden)

So basically it is up to your code to keep or reset the hidden/cell states between batches.
There is a tutorial with more details http://pytorch.org/tutorials/beginner/nlp/sequence_models_tutorial.html 255