Hi,
There will be a small difference in most cases I think.
If you set every requires_grad to False. After each op, it will check whether the output requires gradients.
If you set the no_grad mode, then it won’t even try.