Sorry for the late follow-up.
Yes, you are right. If you want to call backward on the loss, you shouldn’t wrap the forward pass in a torch.no_grad block.
This is useful for validation, test, inference, where no backward will be called and thus the intermediate variables can be freed to save memory.