The canonical Pytorch dimension ordering is (N, C, **) where ** is shape dimensions. For a sequence that gives (N, C, L); for an image (N, C, H, W), etc etc.
LSTM, GRU are a little special because it’s much more efficiently for those to run when their batch dimension isn’t first.