I got a question though. Why should a hidden state require the grad? In the example they pass it in a detached state and hence they are not going to be part of the graph. The hidden state is not supposed to be a parameter to compute derivatives agains right? I’d rather initialize it with requires_grad=False no?