The LSTM network is a type of Recurrent Neural network used for detecting and recognizing sequencial patterns for some given time steps or time series data.
So except the rewards of u RL environment are gotten in a sequencial manner of exploration / exploitation then I suggest u use a different architecture
Anyways even without all these details I’ll still suggest u change ur architecture just to check if it’s really from the network or from sth else u did or failed to do.
Just saying