Answers.Body: <p>Binary cross-entropy loss is a specific case for cross-entropy loss. Theoretically, one can also use the normal cross-entropy loss for binary classification. Binary cross-entropy is probably computationally a lot faster than the general cross-entropy loss and is simpler to understand. However, it does exactly the same as cross-entropy loss if performed on two classes.</p>
  <p>So the only question left is: 1 or 2 output nodes for binary classification? If you use a sigmoid activation for the variant with one output node and the softmax activation for the variant with two output nodes, there is mathematically/theoretically very little difference. One could argue that there are more weights in the network with two output nodes, assuming the rest of the architecture is the same, but that is about it.</p>
  <p><a href="https://stats.stackexchange.com/questions/207049/neural-network-for-binary-classification-use-1-or-2-output-neurons">Similar question with more mathematical answers on stats SE</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-25T07:27:31.803Z
Answers.LastActivityDate: 2023-01-25T07:27:31.803Z
Answers.UserDisplayName: Robin van Hoorn
Body: <p>Let's say I have a binary classification problem and I want to solve it by means of FC neural net. So which approach will be correct: 1) define the last layer of NN like this <code>linear(h, 1)</code> and use Binary Cross entropy 2) define the last layer of NN like this <code>linear(h, 2)</code> and use Cross entropy. It seems for me that these two approaches are similar. Is it correct?</p>
  <p>P.S. This question is more theoretical one, so no any technicalities related to particular framework.</p>
  
Tags: <neural-networks><deep-learning><classification><objective-functions><cross-entropy>
Title: What is the correct loss function for binary classification: Cross entropy or Binary cross entropy?
UserDisplayName: dmasny
ThreadText: # What is the correct loss function for binary classification: Cross entropy or Binary cross entropy?
  
  (Asked by: username_0 on 2023-01-25)
  
  Let's say I have a binary classification problem and I want to solve it by means of FC neural net. So which approach will be correct: 1) define the last layer of NN like this `linear(h, 1)` and use Binary Cross entropy 2) define the last layer of NN like this `linear(h, 2)` and use Cross entropy. It seems for me that these two approaches are similar. Is it correct?
  
  P.S. This question is more theoretical one, so no any technicalities related to particular framework.
  
  **Tags:** neural-networks, deep-learning, classification, objective-functions, cross-entropy
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-01-25. Score: 1 | Accepted answer
  
  Binary cross-entropy loss is a specific case for cross-entropy loss. Theoretically, one can also use the normal cross-entropy loss for binary classification. Binary cross-entropy is probably computationally a lot faster than the general cross-entropy loss and is simpler to understand. However, it does exactly the same as cross-entropy loss if performed on two classes.
  
  So the only question left is: 1 or 2 output nodes for binary classification? If you use a sigmoid activation for the variant with one output node and the softmax activation for the variant with two output nodes, there is mathematically/theoretically very little difference. One could argue that there are more weights in the network with two output nodes, assuming the rest of the architecture is the same, but that is about it.
  
  [Similar question with more mathematical answers on stats SE](https://stats.stackexchange.com/questions/207049/neural-network-for-binary-classification-use-1-or-2-output-neurons).
  
  
  ---
  
  