Answers.Body: <p>Yes, you should use an activation function that match the range of your ground truth labels, or the other way around, i.e. apply a normalization function to the labels to match your activation function.</p>
  <p>If the range of model predictions and target differ in range, the model might still learn but the convergence will be slower, since you're basically allowing a broader range or loss values. With both predictions and labels within range [0, 1] for example, the difference between a prediction and its label will also be in the same range. If you allow predictions to be in range [-1, 1] then you could have errors larger than 1, for example <span class="math-container">$p=-1, t=1, |t-p|=2$</span>.</p>
  <p>The decision about mapping the predictions to the targets range or the way around depends on the specific case you tackling. Sigmoid and tanh for example behave differently with respect to the gradients, the latter producing larger gradients, as you can see from the derivatives comparisons. So if you want larger gradients you might want to keep the tanh and map you targets to the range [-1,1]</p>
  <p><a href="https://i.sstatic.net/EgrnQ.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/EgrnQ.png" alt="enter image description here" /></a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-05-05T09:19:48.607Z
Answers.LastActivityDate: 2022-05-05T09:19:48.607Z
Answers.UserDisplayName: Edoardo Guerriero
Answers.Body: <p>If you have already trained your model to predict either -1 or 1, and now your target variable is either 0 or 1, if -1 predicted by your model is the same as 0 in your target variable, then there are two ways you can do this:</p>
  <ol>
  <li>Use a postprocessing function, which maps the models output, -1 becomes 0, and 1 remains as it is</li>
  <li>Replace your target labels, 0 becomes -1, and 1 remains as it is</li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-05-05T09:12:35.837Z
Answers.LastActivityDate: 2022-05-05T09:12:35.837Z
Answers.UserDisplayName: MD Mushfirat Mohaimin
Body: <p>Suppose I use a <em>tansig</em> activation function in the output layer of an artificial neural network giving me outputs in the range <span class="math-container">$[-1,1]$</span> and my model is applied to a binary classification problem, should my target labels be -1 and 1 or 0 and 1? I've always used 0 and 1, but now I'm questioning this.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-05-05T08:23:03.960Z
Comments.Text: Why are you not using `sigmoid/softmax` for binary classification?
Comments.UserDisplayName: hanugm
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-05-05T14:30:19.130Z
Comments.Text: Would you recommend this? Maybe I should do that. Are there any guidelines available in terms of suitable activation functions for the hidden layers too?
Comments.UserDisplayName: Rowan Barua
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-05-05T14:54:27.193Z
Comments.Text: Yeah, In general, `relu` for hidden layers and `sigmoid` for the output layer.
Comments.UserDisplayName: hanugm
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-05-05T15:08:06.473Z
Comments.Text: Amazing!! Thank you! Seems to have improved results massively by replacing tansig with sigmoid. Now I'll try replacing my tansig hidden layer functions with relu.
Comments.UserDisplayName: Rowan Barua
Tags: <neural-networks><activation-functions><labels>
Title: Should the range of target values match the range of activation function used in the output layer?
UserDisplayName: Rowan Barua
ThreadText: # Should the range of target values match the range of activation function used in the output layer?
  
  (Asked by: username_0 on 2022-05-05)
  
  Suppose I use a *tansig* activation function in the output layer of an artificial neural network giving me outputs in the range $[-1,1]$ and my model is applied to a binary classification problem, should my target labels be -1 and 1 or 0 and 1? I've always used 0 and 1, but now I'm questioning this.
  
  **Tags:** neural-networks, activation-functions, labels
  > **Comments:**
  > **username_1**: Why are you not using `sigmoid/softmax` for binary classification?  
  > **username_0**: Would you recommend this? Maybe I should do that. Are there any guidelines available in terms of suitable activation functions for the hidden layers too?  
  > **username_1**: Yeah, In general, `relu` for hidden layers and `sigmoid` for the output layer.  
  > **username_0**: Amazing!! Thank you! Seems to have improved results massively by replacing tansig with sigmoid. Now I'll try replacing my tansig hidden layer functions with relu.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2022-05-05. Score: 0 | Accepted answer
  
  Yes, you should use an activation function that match the range of your ground truth labels, or the other way around, i.e. apply a normalization function to the labels to match your activation function.
  
  If the range of model predictions and target differ in range, the model might still learn but the convergence will be slower, since you're basically allowing a broader range or loss values. With both predictions and labels within range [0, 1] for example, the difference between a prediction and its label will also be in the same range. If you allow predictions to be in range [-1, 1] then you could have errors larger than 1, for example $p=-1, t=1, |t-p|=2$.
  
  The decision about mapping the predictions to the targets range or the way around depends on the specific case you tackling. Sigmoid and tanh for example behave differently with respect to the gradients, the latter producing larger gradients, as you can see from the derivatives comparisons. So if you want larger gradients you might want to keep the tanh and map you targets to the range [-1,1]
  
  [![enter image description here](https://i.sstatic.net/EgrnQ.png)](https://i.sstatic.net/EgrnQ.png)
  
  
  ---
  
  ### Answer by username_3 on 2022-05-05. Score: 0
  
  If you have already trained your model to predict either -1 or 1, and now your target variable is either 0 or 1, if -1 predicted by your model is the same as 0 in your target variable, then there are two ways you can do this:
  
  1. Use a postprocessing function, which maps the models output, -1 becomes 0, and 1 remains as it is
  2. Replace your target labels, 0 becomes -1, and 1 remains as it is
  
  
  ---
  
  