While the shapes and maths work out in your case (so the answer to your question is could be just “yes”), the general answer is a bit more elaborate:
As you note, autograd uses backpropagation to track the Jacobian-vector-product. However, if you want to interpret this literally as a matrix-vector-product of the Jacobian with the vector, PyTorch  - as would be usual for systems implementing this - does not follow the “mathematical shapes” (vector and matrix) literally.
In backpropagation, we compute the JVP of a composition f ○ g at an input x as D(f ○ g) v = Df (Dg v). Let us say that y = f(x) and z = g(y).
Now v would always be a column vector here as would (Dg v). In PyTorch, however, the convention is that this conceptual column vector is represented by rearranging the entries to the shape of the (intermediate) output, so v will be represented as a tensor of the same shape shape as z and (Dg v) would have the same shape as y. The matrices themselves are typically not explicit either (with matrix multiplication being the big exception), because this would be very inefficient:
Consider a function f multiplying a 3 x 3 x 3 tensor by two. The output is again 3 x 3 x 3 so the vector v in the JVP Df v would be 27 entries So would the JVP itself, and the matrix Df would be a 27 x 27 matrix with 2 on the diagonal and zeros elsewhere, but of course, autograd would not compute it by building that matrix but would just multiply v with 2 to apply Df and get (Df v) = 2v.
If we were maths-braggards we might alternatively describe this as not operating on vectors but on elements in the dual space of the domain and image space (as a finite-dimensional vector spaces over the reals) which we identify with the primal space and the Jacobian D being a contravariant functor mapping each function f : X → Y to Df : D(Y)=Y → D(X)=X (note that because we operate on the dual spaces, we pull back the dual space element using the more conventional linear map between the tangent spaces from, say, functional analysis), and there will be no vectors and instead congratulations for the clever use of category theory in programming. But in the end while it might be good to know that there is some more fancy abstract stuff in the background rather than everything being a fragile ad-hoc construction this doesn’t add that much value here and we are not maths-braggards, are we?
Best regards
Thomas