Solved by vdw in post #2 
              
              
                If your remove batch_first=True it’s of course batch_first=False by default. In this case you would need to change  out = self.fc(out[:, -1, :]) to out = self.fc(out[-1]) 
I don’t what you’re trying to learn and how your data looks like, but x = x.reshape(-1, INPUT_SIZE, MAX_STRING_SIZE) looks a bu…