Answers.Body: <p>Q learning predicts the action value, <span class="math-container">$q(s, a)$</span> for taking action <span class="math-container">$a$</span> in state <span class="math-container">$s$</span>. The action value is usually the discounted sum of all future rewards. In general it can take any scalar value.</p>
  
  <p>DQN uses a neural network to approximate <span class="math-container">$q(s, a)$</span>. Although you might use this to select an action (thus think of the problem as a <em>classification</em>), the NN has to perform <em>regression</em> to predict the action values.</p>
  
  <p>It is most common to use a linear final layer, and mean squared error loss in DQN, to match this regression task. So yes, you will find most control DQNs will make the same choice as the cartpole example that you are looking at.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-08T06:54:12.597Z
Answers.LastActivityDate: 2019-04-08T07:18:07.650Z
Answers.LastEditDate: 2019-04-08T07:18:07.650Z
Answers.UserDisplayName: Neil Slater
Body: <p>I've been reading a lot of tutorials on DQNs for cartpole. In many of them, they have the funnel layer of the neural net be a linear activation.
  Why is this?
  Is it just a choice made by the implementer?
  Is this Choice specific to cartpole, or do most control task dqns use it?
  Thanks. </p>
  
Tags: <neural-networks><dqn>
Title: Why do DQNs use linear activations on cartpole?
UserDisplayName: axon
ThreadText: # Why do DQNs use linear activations on cartpole?
  
  (Asked by: username_0 on 2019-04-08)
  
  I've been reading a lot of tutorials on DQNs for cartpole. In many of them, they have the funnel layer of the neural net be a linear activation.
  Why is this?
  Is it just a choice made by the implementer?
  Is this Choice specific to cartpole, or do most control task dqns use it?
  Thanks.
  
  **Tags:** neural-networks, dqn
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-04-08. Score: 2 | Accepted answer
  
  Q learning predicts the action value, $q(s, a)$ for taking action $a$ in state $s$. The action value is usually the discounted sum of all future rewards. In general it can take any scalar value.
  
  DQN uses a neural network to approximate $q(s, a)$. Although you might use this to select an action (thus think of the problem as a *classification*), the NN has to perform *regression* to predict the action values.
  
  It is most common to use a linear final layer, and mean squared error loss in DQN, to match this regression task. So yes, you will find most control DQNs will make the same choice as the cartpole example that you are looking at.
  
  
  ---
  
  