Answers.Body: <p>Not sure if you're using TensorFlow-Agents for this, but if you are, there's some nice functionality for this built out using the <code>ActorDistributionRnnNetwork</code> and <code>ValueDistributionRnnNetwork</code> that I've found to be very helpful.  The LSTM architecture is used in both the actor and critic networks.  See <a href="https://www.tensorflow.org/agents/api_docs/python/tf_agents/networks/actor_distribution_rnn_network/ActorDistributionRnnNetwork" rel="nofollow noreferrer">this page</a> for more information - hope this helps!  </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-30T04:50:57.687Z
Answers.LastActivityDate: 2020-04-30T04:50:57.687Z
Answers.UserDisplayName: Ryan S
Body: <p>What is the best way of using the LSTM layer in PPO architecture? 
  Should I use them in the first layer of both actor and critic, or use them just before the final layer of these networks? 
  Should I feed the architecture with a stack of states (the state stacked with the k previous states)?</p>
  
Tags: <deep-learning><reinforcement-learning><long-short-term-memory><actor-critic-methods><proximal-policy-optimization>
Title: How to use the LSTM layer in PPO architecture?
UserDisplayName: Sanhood
ThreadText: # How to use the LSTM layer in PPO architecture?
  
  (Asked by: username_0 on 2019-07-28)
  
  What is the best way of using the LSTM layer in PPO architecture?
  Should I use them in the first layer of both actor and critic, or use them just before the final layer of these networks?
  Should I feed the architecture with a stack of states (the state stacked with the k previous states)?
  
  **Tags:** deep-learning, reinforcement-learning, long-short-term-memory, actor-critic-methods, proximal-policy-optimization
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-04-30. Score: 0
  
  Not sure if you're using TensorFlow-Agents for this, but if you are, there's some nice functionality for this built out using the `ActorDistributionRnnNetwork` and `ValueDistributionRnnNetwork` that I've found to be very helpful. The LSTM architecture is used in both the actor and critic networks. See [this page](https://www.tensorflow.org/agents/api_docs/python/tf_agents/networks/actor_distribution_rnn_network/ActorDistributionRnnNetwork) for more information - hope this helps!
  
  
  ---
  
  