Thank you for your reply.
So, I do not wish to reduce the vector into scalar as I need gradients for each output. I do not quite follow the other method, which is passing the gradients to backward? Can you elaborate?
Basically, assume my network output is y and network parameters are x --> i want dy/dx for each y in batch y. Then I’d like to aggregate them all together. So, when you say pass the gradients, how do I pass them when backward is actually what computes the gradients?
As for accessing gradients, in TF - you can get all gradients for all network parameters in a single object --> grads  = tape.gradients, is not there something similar in pytorch? other than accessing them separately like this : model.layer.weight.grad