Simone256:

In particular, there is no way to parallelize the computation of the Jacobian? I guess there’s not…


There is an experimental way to do it  see the vectorize flag on jacobian here 9 or using the vmap prototype directly in your code doc here 7.

Furthermore, let’s say my vec I give as input in .backward() is just a vector of ones, since I’m interested in the sum of the gradients. Is it better to approach the problem this way, or to sum up the outputs of my vectorial loss before? (This should be equivalent for linearity, but maybe one method is computationally / numerically better…

The two are the same really.
Giving the vector of ones will be marginally faster because you create one less Node in the graph, but tha won’t make any measurable difference really.