Answers.Body: <p>A network with ReLU activation can predict negative values; we put ReLU between the hidden layers but return the output of the final layer without any activation function, or with a linear activation function to scale the output.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-10T07:17:45.347Z
Answers.Text: It is worth mentioning that some kind of non-linear activation function is *necessary* for hidden layers to be useful. Only in the output layer should be made linear in most cases (I hesitate to say *all* because although I cannot think of any reason, someone will probably know of a case where it is useful to have a linear hidden layer).
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-10T00:01:41.127Z
Answers.LastActivityDate: 2022-06-10T02:52:01.400Z
Answers.LastEditDate: 2022-06-10T02:52:01.400Z
Answers.UserDisplayName: Lee Reeves
Body: <p>The creation of negative rewards leads to the chance of Q-values being negative. However, networks with relu or sigmoid activations, just cannot predict negative values. This will lead to a case where erroneous Q-values are being predicted. Is my understanding correct?</p>
  
Tags: <reinforcement-learning><dqn><rewards><relu><sigmoid>
Title: Does it make sense to provide a DQN with negative rewards for a network with relu and sigmoid activations?
UserDisplayName: desert_ranger
ThreadText: # Does it make sense to provide a DQN with negative rewards for a network with relu and sigmoid activations?
  
  (Asked by: username_0 on 2022-06-09)
  
  The creation of negative rewards leads to the chance of Q-values being negative. However, networks with relu or sigmoid activations, just cannot predict negative values. This will lead to a case where erroneous Q-values are being predicted. Is my understanding correct?
  
  **Tags:** reinforcement-learning, dqn, rewards, relu, sigmoid
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-06-10. Score: 2 | Accepted answer
  
  A network with ReLU activation can predict negative values; we put ReLU between the hidden layers but return the output of the final layer without any activation function, or with a linear activation function to scale the output.
  
  > **Comments:**
  > **username_2**: It is worth mentioning that some kind of non-linear activation function is *necessary* for hidden layers to be useful. Only in the output layer should be made linear in most cases (I hesitate to say *all* because although I cannot think of any reason, someone will probably know of a case where it is useful to have a linear hidden layer).  
  
  ---
  
  