Yes, in the context of torch.no_grad(), no backwards graph is created so the GPU memory allocation must be smaller than without it. Can you provide more code lines saving outputs tensor? Or, check if any other pre-calculated tensors on GPU were freed properly.