I have tried many methods in our forum.
Such as
torch.backends.cudnn.benchmark = True,
torch.backends.cudnn.enabled = True,
make my input Variables have volatile=True, (High GPU Memory Demand for pytorch?);
and del loss, output at the end of loop (GPU memory consumption increases while training).
But all of them can’t save GPU memory in my experiment!
I use DataParallel, and construct a new network for my specific task.
Could anyone help me with it?