Hi @YichengWang, regarding what you said in 1, looks like after a few passes, pytorch will do the deletion automatically, is this confirmed somewhere in the pytorch documentation?
Also what is the motivation that we need to tune batch size when training a dnn model?
Is it because we want to use 100% of the GPU memory so that it can speed up the training process?
What if I don’t use 100% of the GPU, but say 80%, then my model would still converge, but it will take more batches to finish the training?