Hi,
You are thinking correctly! In your example, your input is not a leaf variable so no grad will be accumulated for it which is the goal of your code too.



 tylerschuessler:

I thought gradients were supposed to accumulate in leaf_variables and this could only happen if requires_grad = True.


For instance, weights and biases of layers such as conv and linear are leaf variables and require grad and when you do backward, grads will be accumulated for them and optimizer will update those leaf variables. So, if you want to compute gradients with respect to your INPUTS too (which can be used to UPDATE INPUTS), like the weights, you need to enable grads for them and make them leaf.
For example, in your code if you add below line after input = torch.randn(1, 1, 32, 32), you can get grads of loss w.r.t. inputs:
input = input.clone().detach().requires_grad_(True)

Bests