Remember to use
model.eval()
with torch.set_grad_enabled(False):
    # your "inference" code here

otherwise your inference mode will create a huge graph in the background for autograd (which will keep growing and growing, since you don’t use .backward() during “inference”).