Hey there,
I am currently trying to build a RNN to detect certain events in a video input stream.
Let’s say the RNN rolls out over a given input sequence and produces an output for each timestep t, but during training I want to backpropagate only after each 4th timestep t_i+3.
Do I have to sum up all the losses t_i, t_i+1, t_i+2 and t_i+3 and run backprop on this sum or do I only have to run backprop on the loss of timestep t_i+3?
My understanding is, when I run backprop on the sum of the losses from t_i to t_4 it is like running backprop after each timestep, is this assumption correct?
Here an image, the pink backpropagation is what I would like to achieve.

IMG_8954.JPG4032×3024 1.7 MB