5had3z:

torch.cuda.empty_cache()


It works and .cpu() is really able to save memory. However, the maximum memory cost is still high. With RaLo4’s help, I know that using



 RaLo4:

running_loss += loss.item()


saves a lot of memory.