I’ve tried adding it as below, but there has been no difference. I also tried gc.collect just to see if there was a variable being held within memory with no difference.
mem_a21 = torch.cuda.memory_summary(device, True)

forward = forward.to("cpu")
torch.cuda.empty_cache()

mem_a3 = torch.cuda.memory_summary(device, True)

I’m really unsure of where to debug from this point forward… I’ve also tried forward.cpu() and torch.device("cpu") so I don’t think it’s specifically an issue with the .to().