Did either of you get to the bottom of this? I get the same issue. Whatever the input the LSTM it returns the same output for each row of the batch. The only difference is the model trains correctly. It is only when it is reloaded and used for predictions this is an issue. Running the LSTM using batch first.