torch.cuda.empty_cache() would free the cached memory so that other processes could reuse it.
However, if you are using the same Python process, this won’t avoid OOM issues and will slow down the code instead.
Based on the reported issue I would assume that you haven’t deleted all references to the model, activations, optimizers, etc. so that some tensors are still alive.