Thanks a lot for your answer, it makes sense now. If I understand correctly, grad_fns will only be evaluated with respect to tensors with requires_grad=True, and that’s how autograd will differentiate the expression wrt the parameters and not the data (for example) ?
Also, this led to torchviz working well, so I could make sure that my model was computing precisely what I wanted.