Oh, wait a moment.
x = x[-1,:,:] looks fishy.
Since you are using batch_first=True, you would slice the tensor in the batch dimension, thus only using the last sample?
If you want to use the last output in the seq dimension, you would have to index the tensor in dim1.
Could you print the shape of output, please?