Body: <p>I am building a neural network for which I am using the sigmoid function as the activation function for the single output neuron at the end. Since the sigmoid function is known to take any number and return a value between 0 and 1, this is causing division by zero error in the back-propagation stage, because of the derivation of cross-entropy. I have seen over the internet it is advised to use a sigmoid activation function with a cross-entropy loss function.</p>
  <p>So, how this error is solved?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-06-02T05:36:30.327Z
Comments.Text: it's not quite clear what you are asking , how will it cause division by zero error? can you describe more about the model?
Comments.UserDisplayName: thecomplexitytheorist
Tags: <neural-networks><backpropagation><cross-entropy><sigmoid><numerical-algorithms>
Title: How is division by zero avoided when implementing back-propagation for a neural network with sigmoid at the output neuron?
UserDisplayName: Dimitry
ThreadText: # How is division by zero avoided when implementing back-propagation for a neural network with sigmoid at the output neuron?
  
  (Asked by: username_0 on 2018-06-02)
  
  I am building a neural network for which I am using the sigmoid function as the activation function for the single output neuron at the end. Since the sigmoid function is known to take any number and return a value between 0 and 1, this is causing division by zero error in the back-propagation stage, because of the derivation of cross-entropy. I have seen over the internet it is advised to use a sigmoid activation function with a cross-entropy loss function.
  
  So, how this error is solved?
  
  **Tags:** neural-networks, backpropagation, cross-entropy, sigmoid, numerical-algorithms
  > **Comments:**
  > **username_1**: it's not quite clear what you are asking , how will it cause division by zero error? can you describe more about the model?  
  
  ---
  
  ## 0 answers
  
  