First, I am no expert in  using LSTM cells for video data.
As far as I understand, LSTMs are not perfect at keeping information over many invocations.
However, the question is, whether this is even necessary.
If you want to classify the current situation of your video stream, is history of e.g. more than 2 minutes ago really important? That entirely depends on the video data you are using.
Think about the kind of data and you goal and how much history the model needs to keep.