nn.RNN will use dim1 as the batch dimension by default as shown in the docs (you can change it via batch_first=True). The permute operation moves the axes and swaps dim0 with dim1 so that the following layers will get an input in the shape [batch_size, ...].