Which one is a better practice to detach a tensor without expanding CPU memory too much? x.detach() or x.cpu() ?
I found out that if I perform x.cpu(), my memory expand quickly and soon reach memory limit error in slurm.