Well, this explains why the mlp outperforms the lstm. Conceptually, to solve cartpole, all you need are the instant positions, speeds and accelerations. If you feed them directly (or even if you feed 3 consecutive positions, which allows to compute these informations), you don’t capture any additionnal useful information using a lstm.