PyTorch has new functionality torch.inference_mode as of v1.9 which is “analogous to 6 torch.no_grad… Code run under this mode gets better performance by disabling view tracking and version counter bumps.”
If I am just evaluating my model at test time (i.e. not training), is there any situation where torch.no_grad is preferable to torch.inference_mode? I plan to replace every instance of the former with the latter, and I expect to use runtime errors as a guardrail (i.e. I trust that any issue would reveal itself as a runtime error, and if it doesn’t surface as a runtime error then I assume it is indeed preferable to use torch.inference_mode).
More details on why inference mode was developed are mentioned in the PyTorch Developer Podcast 10.
I asked a similar question elsewhere 2 but it was the wrong forum.