I mean that, if you are expecting to get the 3d output tensor of means via matrix operations, then it is highly likely that you should construct 4d tensor first and then run .mean(axis=3) on it. As a result it would be 3d tensor.
Try to think in that way, which 4d tensor you can construct first