I’m not sure, if you could save more memory than is already saved by using no_grad() (or the inference_mode()) wrapper. These context managers will make sure to avoid storing forward activations, which would be needed to calculate the gradients, so each forward activation should be deleted after its usage. However, since these activations still have to be computed, you might still run OOM.