I know. Hopefully I will be one of them soon too :D. So, during training time I have example videos of vid_length=2048 frames but the max length my RNN rolls out is seq_length=512 frames, because it takes the output from the Conv3D which itself acts as a feature-detector on delta=16 frames so the actual input for the RNN is seq_length=32 (32 * 16 = 512) because I have this Conv3D which ‘summarizes’ 16 frames into one feature map. a timestep t_n to t_n+1 is 16 frames.
So I want to use 32 feature maps (512 frames) for the RNN and then reset it, yes.