Thanks for the response, @floriandonhauser !
About these individual training samples, following some ideas on the blog spot you mentioned.
If I take a video with a duration of 600 seconds (10 minutes), extract samples of 3 seconds, with a stride of 1 second, the number of samples will be:
10*60 -(3 -1) = 596

Since I have 10 videos, I will have 596*10 samples. If I decrease the stride to half a second I approximately double this amount. It sound a good amount for me, especially knowing that maybe I can get some new videos in the future.
But I have one question about this. When this model gets deployed, it will run on a video stream that can be hours long. If I train on small samples (3 seconds), how will the recurrent cells (LSTM cells) learn to keep long term information on memory?
I will try to train the network with these ideas you presented, already helped me a lot!
Thank you very much.