I’m implementing a model in which a CNN model is used to extract feature sequences from videos , and RNN is used to analyze the generated feature sequences, and output a multi label classification result. Frame-wise annotation of actions is not given but only the temporal order  so the model is being trained with nn.CTCLoss.
I want to use sliding window tecnique to generate more video segments (overlapping windows)  to capture dependencies over different timesteps.
Should i use x.unfold( ) in the input video or  in the generated features after the cnn and before the rnn input ?
does it have any effect  in the backward operation ?