hello
Generally when I try to train model, GPU memory overshoot at the beginning and it will converge after that.
from my test, it overshoot at loss.backward()
Is there way to avoid this overshoot? some model get “CUDA out of memory” because of this
thank you in advance!
image868×482 15.8 KB