Torch provides API functional jacobian to calculate jacobian matrix.
In algorithms, like Levenberg-Marquardt, we need to get 1st-order partial derivatives of loss (a vector) w.r.t each weights(1-D or 2-D) and bias.
With the jacobian function, we can easily get:
torch.autograd.functional.jacobian(nn_func, inputs=inputs_tuple, vectorize=True)

It is fast but vectorize requires much memory. So, I am wondering is it possible to get 1st-order derivative explicitly in PyTorch?
i.e., calculate $\partial{L}/\partial{w_{i}}$ and $\partial{L}/\partial{b_{i}}$