Thanks for the answer! Yes, hidden is indeed a tuple of two tensors.
Unfortunately, this does not work and I really dont get why not…still getting the error from above:



 mayool:

RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time


Does the position matter, where state.detach() is called?
At the moment I call detach in the forward() method of my network.
Does it need to be in the training loop?
so i would call
new_hidden = None
for batch in train_loader:
        hidden_state = new_hidden if new_hidden is not None else init_hidden_with_zeros()
        images, labels = batch
        pred, new_hidden = net(images, hidden_state, old_pred)
        new_hidden = tuple(state.detach() for state in new_hidden)
        (...)

so basicially shifting the process of “taking care of the hidden state” from within the forward method into the train loop…
Does that change anything?