Yes, we are using
with torch.no_grad()

for inference.
We are just wondering if reporting inference time GPU memory usage is not a good idea for showing your model’s memory efficiency based on the reasons stated above.