For the question about an algorithm, I think this thread is useful.

  
    
    
    Example code of recurrent policy gradient? reinforcement-learning
  
  
    Is there an example code for recurrent policy gradient ? Will it be simply replacing MLP with RNN ?