Answers.Body: <p>The <a href="https://en.wikipedia.org/wiki/Identity_function" rel="nofollow noreferrer">identity function</a> can be useful in some cases. For example, if you are doing <em>regression</em>, the output of your neural network needs to be a real (or floating-point) number, so you use the identity function. (If you were doing logistic regression or classification, that wouldn't probably be the case). The identity function is also used in the <em>residual networks</em> (see <a href="https://openaccess.thecvf.com/content_cvpr_2016/papers/He_Deep_Residual_Learning_CVPR_2016_paper.pdf#page=2" rel="nofollow noreferrer">figure 1</a>). There are probably other examples of its usage and usefulness.</p>
  <p>Some people may not consider the identity an activation function because it does nothing to the input. However, whether you consider the identity an activation function or not is a <strong>matter of convention</strong>, in the same way that considering a model with only identity (aka linear) functions a real neural network or a perceptron a neural network is a matter of convention.</p>
  <p>I don't think there's still a consensus on this subject. In fact, you will often hear (or see) people say (or write) that they use &quot;no activation (function)&quot; or &quot;linear activation&quot; rather than saying that they use the &quot;identity&quot; (for example, see the documentation for the parameter <code>activation</code> <a href="https://keras.io/api/layers/core_layers/dense/" rel="nofollow noreferrer">here</a>).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-17T16:19:03.620Z
Answers.LastActivityDate: 2021-09-17T16:19:03.620Z
Answers.UserDisplayName: nbro
Body: <p>It is known that the primary purpose of activation functions, used in neural networks, is to introduce non-linearity.</p>
  <p>Then how can the linear activation function, especially the <strong>identity function</strong>, be treated as an activation function?</p>
  <p>Are there any special applications/advantages in using an identity function as I cannot see any such use theoretically?</p>
  
Tags: <neural-networks><activation-functions>
Title: Why identity function is generally treated as an activation function?
UserDisplayName: hanugm
ThreadText: # Why identity function is generally treated as an activation function?
  
  (Asked by: username_0 on 2021-09-16)
  
  It is known that the primary purpose of activation functions, used in neural networks, is to introduce non-linearity.
  
  Then how can the linear activation function, especially the **identity function**, be treated as an activation function?
  
  Are there any special applications/advantages in using an identity function as I cannot see any such use theoretically?
  
  **Tags:** neural-networks, activation-functions
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-09-17. Score: 3
  
  The [identity function](https://en.wikipedia.org/wiki/Identity_function) can be useful in some cases. For example, if you are doing *regression*, the output of your neural network needs to be a real (or floating-point) number, so you use the identity function. (If you were doing logistic regression or classification, that wouldn't probably be the case). The identity function is also used in the *residual networks* (see [figure 1](https://openaccess.thecvf.com/content_cvpr_2016/papers/He_Deep_Residual_Learning_CVPR_2016_paper.pdf#page=2)). There are probably other examples of its usage and usefulness.
  
  Some people may not consider the identity an activation function because it does nothing to the input. However, whether you consider the identity an activation function or not is a **matter of convention**, in the same way that considering a model with only identity (aka linear) functions a real neural network or a perceptron a neural network is a matter of convention.
  
  I don't think there's still a consensus on this subject. In fact, you will often hear (or see) people say (or write) that they use "no activation (function)" or "linear activation" rather than saying that they use the "identity" (for example, see the documentation for the parameter `activation` [here](https://keras.io/api/layers/core_layers/dense/)).
  
  
  ---
  
  