Body: <p>In the paper <a href="https://arxiv.org/abs/1507.06527" rel="nofollow noreferrer">Deep Recurrent Q-Learning for Partially Observable MDPs</a>, the DRQN is described as DQN with the first <em>post-convolutional</em> fully-connected layer replaced by a recurrent LSTM.</p>
  
  <p>I have DQN implementation with only two dense layers. I want to change this into DRQN with the first layer as an LSTM and leave the second dense layer untouched. If I understood correctly, I would also need to change the input data appropriately. </p>
  
  <p>Are there any other things that need to be modified in order to make DRQN work?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-09T17:18:32.997Z
Comments.Text: Maybe your network cannot be considered the same as (or equivalent to) the DRQN if you also don't have the convolutional layers, even though the convolutional layers were mainly needed in the DQN paper because the input were images. Yours will be a network with a recurrent layer followed by a dense layer. What is the type of your input? Anyway, the Q part in DQN and DRQN only refers to the Q-learning algorithm. We could have called these networks simply deep networks or CNNs or RNNs. We just use the letter Q in order to contextualise the usage of these networks, which have nothing special.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-10T08:32:39.480Z
Comments.Text: Thanks for the clarification. I did not realize DRQN is a general term for a deep network with some recurrent layer and uses Q-learning. I thought it referred to the specific implementation.
Comments.UserDisplayName: Savco
Tags: <reinforcement-learning><ai-design><long-short-term-memory><dqn><deep-rl>
Title: What can be considered a deep recurrent neural network?
UserDisplayName: Savco
ThreadText: # What can be considered a deep recurrent neural network?
  
  (Asked by: username_0 on 2019-04-08)
  
  In the paper [Deep Recurrent Q-Learning for Partially Observable MDPs](https://arxiv.org/abs/1507.06527), the DRQN is described as DQN with the first *post-convolutional* fully-connected layer replaced by a recurrent LSTM.
  
  I have DQN implementation with only two dense layers. I want to change this into DRQN with the first layer as an LSTM and leave the second dense layer untouched. If I understood correctly, I would also need to change the input data appropriately.
  
  Are there any other things that need to be modified in order to make DRQN work?
  
  **Tags:** reinforcement-learning, ai-design, long-short-term-memory, dqn, deep-rl
  > **Comments:**
  > **username_1**: Maybe your network cannot be considered the same as (or equivalent to) the DRQN if you also don't have the convolutional layers, even though the convolutional layers were mainly needed in the DQN paper because the input were images. Yours will be a network with a recurrent layer followed by a dense layer. What is the type of your input? Anyway, the Q part in DQN and DRQN only refers to the Q-learning algorithm. We could have called these networks simply deep networks or CNNs or RNNs. We just use the letter Q in order to contextualise the usage of these networks, which have nothing special.  
  > **username_0**: Thanks for the clarification. I did not realize DRQN is a general term for a deep network with some recurrent layer and uses Q-learning. I thought it referred to the specific implementation.  
  
  ---
  
  ## 0 answers
  
  