Make sure you are not storing unnecessary data, such as the computation graphs by appending the output, loss, etc. of the model in e.g. a list. If you only care about the inference results, you could also wrap the model in a with torch.no_grad() block, if that isn’t already used. Besides that I cannot see any obvious issues in the posted code, but it also doesn’t show a lot.