Solved by LeviViana in post #2 
              
              
                CartPole is actually a very simple problem. Just by using a simple random search you can get more than 200 reward, take a look at this. 
RNNs should outperform MLP’s when the temporal/sequential information is really relevant. In the case of CarPole, you actually don’t care that much about what happ…