Yes, you can wrap any code with torch.no_grad(), where you are sure you won’t need to call backward.
The with statement makes sure to avoid storing intermediate activations in the forward pass, thus saving some memory, and is usually done during evaluation.