Thank you for your answer! I use shared GPU in a lab. Therefore, if the memory use varies in forward pass and backward pass, I am concern labmates will have other jobs in the same GPU when its memory usage is low, but it may be out of memory when it comes back to backward pass. Is there any solution for it?