Answers.Body: <p>Yes, if there's no activation function in the last layer, the weights could simply be negative there, so the network would multiply a positive value with a negative weight, therefore outputting a negative value.</p>
  <p>There is still an activation function, but it is the identity.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-20T14:45:18.817Z
Answers.LastActivityDate: 2020-09-20T14:45:18.817Z
Answers.UserDisplayName: N. Kiefer
Answers.Body: <p>I guess you are using NN for Regresions.
  In the most common aplication a scale of the outputs is implemented.
  This is recommended. Specialy if you have more than one output with diferent scales.
  Otherwise, you will remunerate the neural network for correcting the error of one variable over the other.
  If you still want to avoid a scale of the outputs. Yes. You can use the identity function in the output layer or a linear function (tha same with different slope).
  The weights and bias of some conections will become negative and the hidden neurons are going to work as always.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-21T18:19:51.063Z
Answers.LastActivityDate: 2020-09-21T18:19:51.063Z
Answers.UserDisplayName: Joaquin Torrens
Body: <p>I know that if you use an ReLU activation function at a node in the neural network, the output of that node will be non-negative. I am wondering if it is possible to have a negative output in the final layer, provided that you do not use any activation functions in the final layer, and all the activation functions in the previous hidden layers are ReLU?</p>
  
Tags: <neural-networks><relu>
Title: Is it possible to have a negative output using only ReLU activation functions, but not in the final layer?
UserDisplayName: anonuser01
ThreadText: # Is it possible to have a negative output using only ReLU activation functions, but not in the final layer?
  
  (Asked by: username_0 on 2020-09-20)
  
  I know that if you use an ReLU activation function at a node in the neural network, the output of that node will be non-negative. I am wondering if it is possible to have a negative output in the final layer, provided that you do not use any activation functions in the final layer, and all the activation functions in the previous hidden layers are ReLU?
  
  **Tags:** neural-networks, relu
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2020-09-20. Score: 2
  
  Yes, if there's no activation function in the last layer, the weights could simply be negative there, so the network would multiply a positive value with a negative weight, therefore outputting a negative value.
  
  There is still an activation function, but it is the identity.
  
  
  ---
  
  ### Answer by username_2 on 2020-09-21. Score: 0
  
  I guess you are using NN for Regresions.
  In the most common aplication a scale of the outputs is implemented.
  This is recommended. Specialy if you have more than one output with diferent scales.
  Otherwise, you will remunerate the neural network for correcting the error of one variable over the other.
  If you still want to avoid a scale of the outputs. Yes. You can use the identity function in the output layer or a linear function (tha same with different slope).
  The weights and bias of some conections will become negative and the hidden neurons are going to work as always.
  
  
  ---
  
  