Answers.Body: <p>Although it's true that if you use certain trigonometric functions, such as the tangent, you could have numerical problems (as suggested <a href="https://ai.stackexchange.com/a/22108/2444">in this answer</a>), that's not the only reason for not using trigonometric functions.</p>
  <p>Trigonometric functions are periodic. In general, we may not want to convert a non-periodic function to a periodic one. To be more concrete, let's suppose we use the sine function as the activation function of the output neurons of a neural network. Assuming only one input, if the input to any of those output neurons is <span class="math-container">$360k$</span>, for any integer <span class="math-container">$k$</span>, the result will always be <span class="math-container">$0$</span>, but that may not be desirable.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-23T13:45:26.567Z
Answers.LastActivityDate: 2020-06-23T13:45:26.567Z
Answers.UserDisplayName: nbro
Answers.Body: <p>The main reason why the sigmoid function is used is because it 'does not blow up' since it is between 0 and 1 always. As for the relu it is used because it is computationally cheap and even resolves the problem of vanishing gradient(and hence used more often than sigmoid).</p>
  <p>So a function like tan(x) will blow up for certain values of x. This can cause a problem of exploding gradients. So, I believe for this reason tan(x) cannot be a good non-linearity to be used.</p>
  <p>As for any other function, it is more because of the results that we have gotten over the years and sigmoid and relu have been promising.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-23T13:20:36.770Z
Answers.LastActivityDate: 2020-06-23T13:20:36.770Z
Answers.UserDisplayName: encrypted_soul
Body: <p>Why don't we use a trigonometric function, such as <span class="math-container">$\tan(x)$</span>, where <span class="math-container">$x$</span> is an element of the interval <span class="math-container">$[0,pi/2)$</span>, instead of the sigmoid function for the output neurons (in the case of classification)?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-06-23T11:03:09.223Z
Comments.Text: Are you only interested in classification or also regression?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-06-23T11:04:38.917Z
Comments.Text: In classification. I'm curious as to why we fit the data using a sigmoid function.
Comments.UserDisplayName: AC18
Tags: <neural-networks><classification><activation-functions>
Title: Why don't we use trigonometric functions for the output neurons?
UserDisplayName: AC18
ThreadText: # Why don't we use trigonometric functions for the output neurons?
  
  (Asked by: username_0 on 2020-06-23)
  
  Why don't we use a trigonometric function, such as $\tan(x)$, where $x$ is an element of the interval $[0,pi/2)$, instead of the sigmoid function for the output neurons (in the case of classification)?
  
  **Tags:** neural-networks, classification, activation-functions
  
  > **Comments:**
  > **username_1**: Are you only interested in classification or also regression?  
  > **username_0**: In classification. I'm curious as to why we fit the data using a sigmoid function.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2020-06-23. Score: 0 | Accepted answer
  
  Although it's true that if you use certain trigonometric functions, such as the tangent, you could have numerical problems (as suggested [in this answer](https://ai.stackexchange.com/a/22108/2444)), that's not the only reason for not using trigonometric functions.
  
  Trigonometric functions are periodic. In general, we may not want to convert a non-periodic function to a periodic one. To be more concrete, let's suppose we use the sine function as the activation function of the output neurons of a neural network. Assuming only one input, if the input to any of those output neurons is $360k$, for any integer $k$, the result will always be $0$, but that may not be desirable.
  
  
  ---
  
  ### Answer by username_2 on 2020-06-23. Score: 2
  
  The main reason why the sigmoid function is used is because it 'does not blow up' since it is between 0 and 1 always. As for the relu it is used because it is computationally cheap and even resolves the problem of vanishing gradient(and hence used more often than sigmoid).
  
  So a function like tan(x) will blow up for certain values of x. This can cause a problem of exploding gradients. So, I believe for this reason tan(x) cannot be a good non-linearity to be used.
  
  As for any other function, it is more because of the results that we have gotten over the years and sigmoid and relu have been promising.
  
  
  ---
  
  