I am not sure how jax computes the full Jacobian, but I have always been curious about that. I might do some digging later.
To summarize our discussion above, it sounds like you just want to compute the full Jacobian and use it as a penalty. I posted some code above that computes a Jacobian by invoking autograd.grad multiple times; why doesn’t that work with your use case?