I had the same issue and searched for a solution for a long time. I needed to do two things. Delete the forward return after the call (Or transfer to cpu) and wrap some tensors that were being stored, inside torch.autograd.Variables, (I did tbptt that time). This solution seems to me like a hack, because Variables is marked as deprecated and there surely is another approach to do this better. But I realized after I did that, that I didn’t have to delete anything inside the forward method anymore.
Maybe It will help you solve the problem.
FYI: It would help to know what exactly you are doing.