I mean that the Jacobian matrix is never actually created. And even though this matrix is n x n in size, we can do the dot product with a vector of size n in time linear in n instead of n^2 (using the sparsity of the matrix).
But the logic you talk about above about optimizing matrix products is based on the idea that regular matrix product is n^2 complexity.
I am not saying you cannot do it, but you might not see the benefits you expect.