I guess it’ll be hard to say without seeing the code / data. Also, I’m not sure if LSTM is the right tool given small amount of data.
Either way, to clarify: you train LSTM on train data, predict 0 / 1 labels for the test data,
and then on the test data and predicted labels you train a new, seperate classifier?
Is this also an LSTM? Did you check the number of predicted 0s and 1s? And how many of those 0s are of merged-class-1, and how many belong to merged-class-2?
As a sidenote, you should not train anything on the test data.