Hi @PTNobel,
From my understanding, gradients are calculated via Jacobian-Vector products within PyTorch and so in order to calculate the gradient of all of the outputs we pass a Tensor filled with ones that is the same shape as the output in order for the gradient to be calculated for all components of our output, and for all samples in our batch.
With regards to other values, you can read in more detail in the docs torch.autograd.grad — PyTorch 1.9.0 documentation 1 but there are retain_graph (used to keep a graph if you’re calculating gradients on the same graph multiple times), create_graph (used if you want to calculate higher-order derivatives), and allow_unused which is for inputs that have no gradient.
I’m not a developer so this is just my understanding (which could be wrong, so if it is, anyone please correct me). One thing that might help you understand the use of grad_outputs is to read up in more detail about Jacobian-Vector products and how they’re used to compute gradients in Automatic Differentiation!