You would have to set batch_first=True, if the input uses dim0 as the batch dimension.
Otherwise you would use the input tensor batch dimension as the temporal dimension and the temp. dimension as the batch size.
Note that nn.CrossEntropyLoss expects raw logits so you would also have to remove the nn.Softmax activation from the model.