Body: <p>Is there a method (not a table of recommendations!) that could tell me what activation function to choose if the outputs of the neural network have some interpretation? For example, these can be the mean of some normal distribution, probabilities in multinomial distribution, parameters of the exponential distribution, and so on.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-11-06T02:33:55.827Z
Comments.Text: This is an old question, but I think you should clarify what you really mean by "interpretation". Do you mean that the outputs are **just** "the mean of some normal distribution, probabilities in multinomial distribution, parameters of the exponential distribution", i.e. related to distributions, or maybe something else?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-11-06T10:30:57.270Z
Comments.Text: Yes, please, clarify. NNs are just powerful aproximators. You can interpret the network outputs as you want (and treat them accordingly in the loss_fn). The act_fn are more an architectural design question rather than a information coding/interpretation question.
Comments.UserDisplayName: JVGD
Tags: <neural-networks><activation-functions>
Title: How do I choose the activation function of the output layer of a neural network (based on theoretical motivations)?
UserDisplayName: Josef Ondrej
ThreadText: # How do I choose the activation function of the output layer of a neural network (based on theoretical motivations)?
  
  (Asked by: username_0 on 2019-03-01)
  
  Is there a method (not a table of recommendations!) that could tell me what activation function to choose if the outputs of the neural network have some interpretation? For example, these can be the mean of some normal distribution, probabilities in multinomial distribution, parameters of the exponential distribution, and so on.
  
  **Tags:** neural-networks, activation-functions
  > **Comments:**
  > **username_1**: This is an old question, but I think you should clarify what you really mean by "interpretation". Do you mean that the outputs are **just** "the mean of some normal distribution, probabilities in multinomial distribution, parameters of the exponential distribution", i.e. related to distributions, or maybe something else?  
  > **username_2**: Yes, please, clarify. NNs are just powerful aproximators. You can interpret the network outputs as you want (and treat them accordingly in the loss_fn). The act_fn are more an architectural design question rather than a information coding/interpretation question.  
  
  ---
  
  ## 0 answers
  
  