From LayerNorm — PyTorch 1.8.1 documentation 2,

If a single integer is used, it is treated as a singleton list, and this module will normalize over the last dimension which is expected to be of that specific size.

So, TLDR it doesn’t care if seq len and batch size are permuted as long as the last dim is correct.