Hi Gabriel,
a starting point can be the gradient of the score (or the cross entropy with the predicted label, but that might run into saturation issues) by LSTM input (see Image-Specific Class Saliency Visualisation in Simonyan et al.: Deep inside conv networks 20. You can compute the gradient by using torch.autograd.grad.
Best regards
Thomas