Hi,
based on my understanding, if all leaves nodes (which are usually parameters and inputs) have requires_grad=False, then all intermediate nodes will have requires_grad=False and therefore, the intermediate nodes are also considered leaves as well. So, no gradian is computed.
Furthermore, even if some nodes have requires_grad = True, only the gradient is computed for the necessary intermediate nodes that are connected to the nodes with requires_grad = True.
So, in many models (at least sequential ones) no gradient is computed if requires_grad=False for weights and inputs.
My understanding comes from:PyTorch Autograd Explained - In-depth Tutorial
https://pytorch.org/docs/stable/notes/autograd.html 21
Please let me know if I miss understood something.
Thanks