Answers.Body: <p>I am not into the field of super-resolution, but I think this question applies to general neural network construction.</p>
  <p>Usually, you try to solve a classification problem or a regression problem with your neural network.</p>
  <ul>
  <li><p>For classification, you try to predict probabilities that a specific output corresponds to a specific class. Therefore, every output value should be a probability and therefore have a range between 0 and 1. To achieve this, you usually use a softmax or sigmoid function as your last layer to squash the output between 0 and 1. In addition to this (which is wanted in classification tasks), these functions raise the probability output of likely classes while decreasing the probability of all other unlikely classes (therefore enforcing the network to choose for one specific class over the others).</p>
  </li>
  <li><p>For the regression task, you are not looking for probability values as your output values but instead for real-valued numbers. In such a case, no activation function is wanted, since you want to be able to approximate any possible real value and not probabilities.</p>
  </li>
  </ul>
  <p>So, in the case of super-resolution, I think the generated output is a map where each value corresponds to a pixel value of the super-resolution image. In that case, your pixels are real number values and no probabilities. So, you are solving a regression problem.</p>
  <p>But you could also go with a classification approach, where you have 256 output maps that give probability to each possible pixel value between <span class="math-container">$0$</span> and <span class="math-container">$255$</span>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-03-14T12:17:34.923Z
Answers.LastActivityDate: 2020-11-15T01:19:22.313Z
Answers.LastEditDate: 2020-11-15T01:19:22.313Z
Answers.UserDisplayName: Marcel_marcel1991
Answers.Body: <p>As discussed here:</p>
  
  <p><a href="https://www.researchgate.net/post/What_should_be_my_activation_function_for_last_layer_of_neural_network" rel="nofollow noreferrer">https://www.researchgate.net/post/What_should_be_my_activation_function_for_last_layer_of_neural_network</a>   </p>
  
  <p>Linear is the preferred activation function.</p>
  
  <p>But then linear activation function is equivalent to no activation function at all:</p>
  
  <p><a href="https://datascience.stackexchange.com/questions/13696/lack-of-activation-function-in-output-layer-at-regression">https://datascience.stackexchange.com/questions/13696/lack-of-activation-function-in-output-layer-at-regression</a></p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-02-11T01:12:42.457Z
Answers.LastActivityDate: 2018-02-11T01:12:42.457Z
Answers.UserDisplayName: Peter Teoh
Body: <p>I'm trying to implement some image super-resolution models on medical images. After reading a set of papers, I found that none of the existing models use any activation function for the last layer.</p>
  <p>What's the rationale behind that?</p>
  
Tags: <deep-learning><computer-vision><deep-neural-networks><activation-functions><image-super-resolution>
Title: Why is no activation function used at the final layer of super-resolution models?
UserDisplayName: Saeed
ThreadText: # Why is no activation function used at the final layer of super-resolution models?
  
  (Asked by: username_0 on 2018-02-10)
  
  I'm trying to implement some image super-resolution models on medical images. After reading a set of papers, I found that none of the existing models use any activation function for the last layer.
  
  What's the rationale behind that?
  
  **Tags:** deep-learning, computer-vision, deep-neural-networks, activation-functions, image-super-resolution
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2018-03-14. Score: 7
  
  I am not into the field of super-resolution, but I think this question applies to general neural network construction.
  
  Usually, you try to solve a classification problem or a regression problem with your neural network.
  
  * For classification, you try to predict probabilities that a specific output corresponds to a specific class. Therefore, every output value should be a probability and therefore have a range between 0 and 1. To achieve this, you usually use a softmax or sigmoid function as your last layer to squash the output between 0 and 1. In addition to this (which is wanted in classification tasks), these functions raise the probability output of likely classes while decreasing the probability of all other unlikely classes (therefore enforcing the network to choose for one specific class over the others).
  * For the regression task, you are not looking for probability values as your output values but instead for real-valued numbers. In such a case, no activation function is wanted, since you want to be able to approximate any possible real value and not probabilities.
  
  So, in the case of super-resolution, I think the generated output is a map where each value corresponds to a pixel value of the super-resolution image. In that case, your pixels are real number values and no probabilities. So, you are solving a regression problem.
  
  But you could also go with a classification approach, where you have 256 output maps that give probability to each possible pixel value between $0$ and $255$.
  
  
  ---
  
  ### Answer by username_2 on 2018-02-11. Score: 0
  
  As discussed here:
  
  <https://www.researchgate.net/post/What_should_be_my_activation_function_for_last_layer_of_neural_network>
  
  Linear is the preferred activation function.
  
  But then linear activation function is equivalent to no activation function at all:
  
  <https://datascience.stackexchange.com/questions/13696/lack-of-activation-function-in-output-layer-at-regression>
  
  
  ---
  
  