No problem, hopefully we can find the problem!
You just need to make sure you detach before using the hidden state again. It does not matter if you do it when you store the hidden state in the forward pass or when you retrieve the previous hidden state and feed it together with the new input to the LSTM.
The old_pred tensors need to be detached as well if it is concatenated with tensors in the current forward pass. Because the old_pred tensors still have a computation graph!
# In training loop
old_pred[1] = old_pred[0]
old_pred[0] = pred.unsqueeze(1).detach()

Hopefully it works this way. Gonna be unavailable soon so might take a day for me to respond next time