Ho I just noticed that you update self.hidden in your forward function.
That means that the next iteration will depend on the previous one.
I am not sure how you want to handle this. But you should either .detach() if you want to keep the updated value without having gradients flow back all the way to the previous iteration.
Or if the value of the original hidden state should be fixed to a constant value, then you need to re-initialize it with a Tensor with no history.