Answers.Body: <p>The approximator can be any artificial neural network architecture, including deep fully-connected networks.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-18T14:12:18.063Z
Answers.LastActivityDate: 2019-03-22T09:10:32.753Z
Answers.LastEditDate: 2019-03-22T09:10:32.753Z
Answers.UserDisplayName: Bionic Buffulo
Answers.Body: <p>No. DQN and other deep RL methods work well with fully connected layers. Here's an implementation of DQN which doesn't use CNNs: <a href="http://github.com/keon/deep-q-learning/blob/master/dqn.py" rel="noreferrer">github.com/keon/deep-q-learning/blob/master/dqn.py</a></p>
  
  <p>DeepMind mostly use CNN because they use image as input state, and that because they tried to evaluate performance of their methods vs humans performance. Humane performance is easy to measure at games with image as input state, and that's why CNN based methods present so promptly in RL now. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-15T03:44:37.500Z
Answers.Text: I start thinking, is that because of the meaning of "deep" on DeepRL means "End-to-End without feature engineering" so most people use the term along with CNN?
Answers.UserDisplayName: malioboro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-15T04:18:50.760Z
Answers.Text: @malioboro CNN's are essential for using images an inputs to a neural network. If you have image inputs to your network, then you will probably use a CNN. If you don't have images as inputs to your network, you probably won't (not getting into time-series inputs and the like).
Answers.UserDisplayName: Omegastick
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-14T06:47:16.953Z
Answers.LastActivityDate: 2019-03-15T13:38:47.007Z
Answers.LastEditDate: 2019-03-15T13:38:47.007Z
Answers.UserDisplayName: mirror2image
Body: <p>I read top articles on Google Search about Deep Q-Learning:</p>
  
  <ul>
  <li><a href="https://medium.freecodecamp.org/an-introduction-to-deep-q-learning-lets-play-doom-54d02d8017d8" rel="noreferrer">https://medium.freecodecamp.org/an-introduction-to-deep-q-learning-lets-play-doom-54d02d8017d8</a></li>
  <li><a href="https://skymind.ai/wiki/deep-reinforcement-learning" rel="noreferrer">https://skymind.ai/wiki/deep-reinforcement-learning</a></li>
  <li><a href="https://neuro.cs.ut.ee/demystifying-deep-reinforcement-learning/" rel="noreferrer">https://neuro.cs.ut.ee/demystifying-deep-reinforcement-learning/</a></li>
  <li><a href="https://en.wikipedia.org/wiki/Q-learning#Deep_Q-learning" rel="noreferrer">Q-Learning page on wikipedia</a>*</li>
  </ul>
  
  <p>and then I noticed that they all use CNN as approximator. If deep learning has a broader definition than just CNN, can we use the term "Deep Q-Learning" on our model if we don't use CNN? or is there a more appropriate definition for that kind of Q-Learning model? for example, if my model only using deep fully-connected layer.</p>
  
  <p>*<sub>it doesn't say explicitly Deep RL means CNN on RL, but it uses the DeepMind (that uses CNN) as an example on Deep Q-Learning </sub></p>
  
Tags: <reinforcement-learning><definitions><deep-rl>
Title: Do we have to use CNN for Deep Q Learning?
UserDisplayName: malioboro
ThreadText: # Do we have to use CNN for Deep Q Learning?
  
  (Asked by: username_0 on 2019-03-14)
  
  I read top articles on Google Search about Deep Q-Learning:
  
  * <https://medium.freecodecamp.org/an-introduction-to-deep-q-learning-lets-play-doom-54d02d8017d8>
  * <https://skymind.ai/wiki/deep-reinforcement-learning>
  * <https://neuro.cs.ut.ee/demystifying-deep-reinforcement-learning/>
  * [Q-Learning page on wikipedia](https://en.wikipedia.org/wiki/Q-learning#Deep_Q-learning)\*
  
  and then I noticed that they all use CNN as approximator. If deep learning has a broader definition than just CNN, can we use the term "Deep Q-Learning" on our model if we don't use CNN? or is there a more appropriate definition for that kind of Q-Learning model? for example, if my model only using deep fully-connected layer.
  
  \*it doesn't say explicitly Deep RL means CNN on RL, but it uses the DeepMind (that uses CNN) as an example on Deep Q-Learning
  
  **Tags:** reinforcement-learning, definitions, deep-rl
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2019-03-14. Score: 10 | Accepted answer
  
  No. DQN and other deep RL methods work well with fully connected layers. Here's an implementation of DQN which doesn't use CNNs: [github.com/keon/deep-q-learning/blob/master/dqn.py](http://github.com/keon/deep-q-learning/blob/master/dqn.py)
  
  DeepMind mostly use CNN because they use image as input state, and that because they tried to evaluate performance of their methods vs humans performance. Humane performance is easy to measure at games with image as input state, and that's why CNN based methods present so promptly in RL now.
  
  > **Comments:**
  > **username_0**: I start thinking, is that because of the meaning of "deep" on DeepRL means "End-to-End without feature engineering" so most people use the term along with CNN?  
  > **username_3**: @username_0 CNN's are essential for using images an inputs to a neural network. If you have image inputs to your network, then you will probably use a CNN. If you don't have images as inputs to your network, you probably won't (not getting into time-series inputs and the like).  
  
  ---
  
  ### Answer by username_1 on 2019-03-18. Score: 0
  
  The approximator can be any artificial neural network architecture, including deep fully-connected networks.
  
  
  ---
  
  