Solved by albanD in post #2 
              
              
                Hi, 
There will be a small difference in most cases I think. 
If you set every requires_grad to False. After each op, it will check whether the output requires gradients. 
If you set the no_grad mode, then it won’t even try.