Unfortunately, we do not support doing the vector Jacobian product with multiple vectors at the same time. You have to use a for loop and multiple calls to backward (as is done in the gist I linked above).

Also, the aim of backpropagation is to get this Jacobian.

This is only true when your function has scalar output. If it has multiple outputs, then it only computes vector Jacobian product. This is a limitation to any deep learning framework and Automatic Differentiation in general.
The performance of using this during training is going to be quite high. It will be very dependent on your network, so you will want to try and see how slow it is.