Solved by albanD in post #8 
              
              
                Ho I just noticed that you update self.hidden in your forward function. 
That means that the next iteration will depend on the previous one. 
I am not sure how you want to handle this. But you should either .detach() if you want to keep the updated value without having gradients flow back all the wa…