No problem.
What do you send as a state? Your input should have 2 items, a state (in Atari it’s an image) and a tuple of the LSTM hidden and cell states (hx,cx) from the previous run.
Example from A3C:
        if done: # Initialization
            cx = Variable(torch.zeros(1, 256))
            hx = Variable(torch.zeros(1, 256))
        else: # The hx,cx from the previous iteration
            cx = Variable(cx.data)
            hx = Variable(hx.data)

value, logit, (hx, cx) = model(
                (Variable(state.unsqueeze(0)), (hx, cx)))  # notice what the model receives as inputs.