Answers.Body: <blockquote>
    <p>And following this point, aren't you just as well off with just using the identity function as your output activation function?</p>
  </blockquote>
  
  <p>When someone declares that the output of a neural network layer is <em>linear</em> this is exactly what they mean. It can also be described as "no activation function". </p>
  
  <p>Saying that a NN layer has <em>linear activation</em> is a kind of short-hand for saying "the whole layer is a linear function of its inputs", which is true without adding any activation function, just using the weights and bias. </p>
  
  <p>There is usually no separate linear function applied, and libraries such as <a href="https://keras.io/activations/#linear" rel="nofollow noreferrer">Keras include the term 'linear' only for completeness</a>, or so that the choice can be made explicit in the code, as opposed to an unseen default.</p>
  
  <p>Note that the link to Keras activation definition above says:</p>
  
  <blockquote>
    <p>Linear (i.e. identity) activation function.</p>
  </blockquote>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-09-26T15:54:34.997Z
Answers.LastActivityDate: 2018-09-26T16:00:28.673Z
Answers.LastEditDate: 2018-09-26T16:00:28.673Z
Answers.UserDisplayName: Neil Slater
Body: <p>I have a simple question about the choice of activation function for the output layer in feed-forward neural networks.</p>
  <p>I have seen several codes where the choice of the activation function for the output layer is linear.</p>
  <p>Now, it might well be that I am wrong about this, but isn't that simply equivalent to a rescaling of the weights connecting the last hidden layer to the output layer? And following this point, aren't you just as well off with just using the identity function as your output activation function?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-09-26T15:47:37.723Z
Comments.Text: Do you mean using a linear activation function that is not the identity function? Like f(x)=2x?
Comments.UserDisplayName: Mr. Eivind
Tags: <neural-networks><terminology><activation-functions>
Title: Is a linear activation function (in the output layer) equivalent to an identity function?
UserDisplayName: user13514
ThreadText: # Is a linear activation function (in the output layer) equivalent to an identity function?
  
  (Asked by: username_0 on 2018-09-26)
  
  I have a simple question about the choice of activation function for the output layer in feed-forward neural networks.
  
  I have seen several codes where the choice of the activation function for the output layer is linear.
  
  Now, it might well be that I am wrong about this, but isn't that simply equivalent to a rescaling of the weights connecting the last hidden layer to the output layer? And following this point, aren't you just as well off with just using the identity function as your output activation function?
  
  **Tags:** neural-networks, terminology, activation-functions
  
  > **Comments:**
  > **username_1**: Do you mean using a linear activation function that is not the identity function? Like f(x)=2x?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2018-09-26. Score: 3
  
  > And following this point, aren't you just as well off with just using the identity function as your output activation function?
  
  When someone declares that the output of a neural network layer is *linear* this is exactly what they mean. It can also be described as "no activation function".
  
  Saying that a NN layer has *linear activation* is a kind of short-hand for saying "the whole layer is a linear function of its inputs", which is true without adding any activation function, just using the weights and bias.
  
  There is usually no separate linear function applied, and libraries such as [Keras include the term 'linear' only for completeness](https://keras.io/activations/#linear), or so that the choice can be made explicit in the code, as opposed to an unseen default.
  
  Note that the link to Keras activation definition above says:
  
  > Linear (i.e. identity) activation function.
  
  
  ---
  
  