Solved by rileypsmith in post #4 
              
              
                I ended up figuring this out. After closer reading of the docs, I noticed that it does specify that with batch_first = True, only the input and output tensors are reported with batch first. The initial memory states (h_init and c_init) are still reported with batch second. 
This seems like a bit of …