How to know the number of dimensions of a Jacobian? My question comes from a comment in this question Vector Jacobian product in automatic differentiation
The question states...
$$
t = Wz, \,\,\, z\in \mathbb{R}^{m\times 1}, t \in \mathbb{R}^{n \times 1}, W\in\mathbb{R}^{n \times m}
$$
$$
\frac{\partial t}{\partial z} = W
$$
Which is all good but then a comment states an observation that a different Jacobian, $\frac{\partial t}{\partial W} \in \mathbb{R}^{n \times n \times m}$. I cannot justify to myself wwhy that Jacobian would have three axes. Can anyone explain this?

Well, $t$ is a vector, and $W$ is a matrix. Think about differentiating everything element-wise, i.e.
$$D_{ijk}=\frac{\partial t_i}{\partial W_{jk}}$$
and $D$ automatically becomes a 3D tensor.
