The reason is that the cudnn backend defines the parameter ordering in this way
http://docs.nvidia.com/deeplearning/sdk/cudnn-user-guide/index.html#cudnnGetRNNWorkspaceSize 9
There is a ‘batch_first’ option which you can set to True so that your input tensor can have its first dimension equal to the batch_size.