If my input tensors to the loss function (that require gradient computation) are two matrices and the output is a single value. The grad_output is what exactly? And what should I do with it? The tensors that have to be returned by backward are tensors with the same shape as the input to the loss function, right?
I guess my grad_output is one because there are not partial derivatives needed because it is the loss function.
Is there more documentation on this? I guess this is obvious on the simple cases, but not for more complex functions.