Answers.Body: <p>There are several nice things about using the logistic function and a few drawbacks as well.  I'm not going to discuss all of them, but here is a small synopsis.</p>
  <h1>Nice Properties</h1>
  <p><a href="https://i.sstatic.net/czEqL.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/czEqL.png" alt="enter image description here" /></a></p>
  <h3>For networks which calculate probabilities, it brings the input features into a nice 0 to 1 range.</h3>
  <p>Since the logistic function maps inputs to the range (0,1), it is commonly used as an activation function in neural networks which require some sort of classification.  For example, if we had a convolution neural network (CNN) that classified if an image given to it was a dog or not, it could have an output neuron that returned 0 if the image was not a dog or 1 if it was.  When the CNN is given an image with inputs of varying dimensions and ranges, it's more intuitive to deal with the input features in the realm of 0 to 1 instead of 0 to 255 or whatever input format you're dealing with.</p>
  <h3>It is smooth and differentiable.</h3>
  <p>When performing back propagation, it's much easier to deal with differential functions.  Otherwise you'd have to approximate the gradient which may lead to unfavorable results.</p>
  <h1>A Vanishing Issue</h1>
  <h3>At extreme values, the gradient vanishes.</h3>
  <p>The OP was also hinting that at very positive and very negative values of X, derivative of the logistic function could become close to 0.  For large input values this may hinder learning for extreme values.  If you think of gradient decent, the jump that gradient descent makes is only as strong as its magnitude of the gradient that it requires.  Analytically speaking, this also hurts back propagation because learning is calculated using the chain rule, which ends up multiplying several gradients together.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-02-04T01:05:55.140Z
Answers.LastActivityDate: 2019-02-04T01:05:55.140Z
Answers.LastEditDate: 2020-06-17T09:57:20.710Z
Answers.UserDisplayName: juicedatom
Body: <p>As in sigmoid function when input x is very large or very small the curve is flat that means low gradient descent but when it is in between the slope is more so,
  My question is how this thing helps us in neural network.</p>
  
Tags: <neural-networks>
Title: How sigmoid funtion helps us in reducing error in neural networks?
UserDisplayName: Harshal
ThreadText: # How sigmoid funtion helps us in reducing error in neural networks?
  
  (Asked by: username_0 on 2019-02-03)
  
  As in sigmoid function when input x is very large or very small the curve is flat that means low gradient descent but when it is in between the slope is more so,
  My question is how this thing helps us in neural network.
  
  **Tags:** neural-networks
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-02-04. Score: 2
  
  There are several nice things about using the logistic function and a few drawbacks as well. I'm not going to discuss all of them, but here is a small synopsis.
  
  # Nice Properties
  
  [![enter image description here](https://i.sstatic.net/czEqL.png)](https://i.sstatic.net/czEqL.png)
  
  ### For networks which calculate probabilities, it brings the input features into a nice 0 to 1 range.
  
  Since the logistic function maps inputs to the range (0,1), it is commonly used as an activation function in neural networks which require some sort of classification. For example, if we had a convolution neural network (CNN) that classified if an image given to it was a dog or not, it could have an output neuron that returned 0 if the image was not a dog or 1 if it was. When the CNN is given an image with inputs of varying dimensions and ranges, it's more intuitive to deal with the input features in the realm of 0 to 1 instead of 0 to 255 or whatever input format you're dealing with.
  
  ### It is smooth and differentiable.
  
  When performing back propagation, it's much easier to deal with differential functions. Otherwise you'd have to approximate the gradient which may lead to unfavorable results.
  
  # A Vanishing Issue
  
  ### At extreme values, the gradient vanishes.
  
  The OP was also hinting that at very positive and very negative values of X, derivative of the logistic function could become close to 0. For large input values this may hinder learning for extreme values. If you think of gradient decent, the jump that gradient descent makes is only as strong as its magnitude of the gradient that it requires. Analytically speaking, this also hurts back propagation because learning is calculated using the chain rule, which ends up multiplying several gradients together.
  
  
  ---
  
  