Solved by albanD in post #2 
              
              
                Hi, 
If you only need to backprop the last cycle, you can .detach() the hidden state between each cycle. That way, no gradient will flow back to the previous cycles.