Answers.Body: <ol>
  <li>BCELoss ( Binary Cross Entropy Loss) is used for binary classifier, which is a neural network that have a binary output, 0 or 1. It is not used for multi-output neural network like your case. For that kind of networks, you can use MSELoss or CrossEntropyLoss as your loss for the network. </li>
  <li>For the calculation of BCE, it is shown on pytorch documentation.
  <a href="https://pytorch.org/docs/stable/nn.html#BCELoss" rel="nofollow noreferrer">https://pytorch.org/docs/stable/nn.html#BCELoss</a>
  For across batch, it either sum the loss or take the mean. you can set that through the reduction="" argument. </li>
  <li>As i said, the loss is either summed or taken the mean. Then there will be a single value so pytorch can do backdrop.</li>
  </ol>
  
  <p>Hope I can help you</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-11T04:09:29.853Z
Answers.LastActivityDate: 2019-10-11T04:09:29.853Z
Answers.UserDisplayName: Clement
Body: <p>I've a prediction matrix(P) of dimension 3x3 and one-hot encoded label matrix(L) of dimension 3x3 as shown below.</p>
  
  <pre><code>    |0.5 0.3 0.1|      |1 0 0|
  P = |0.3 0.2 0.1|  L = |0 1 0|
      |0.2 0.5 0.8|      |0 0 1|
  </code></pre>
  
  <p>each column in 'P' corresponds to prediction of a label in 'L'</p>
  
  <ol>
  <li><p>How is the BCELoss calculated using pytorch?, my experimentation by giving these two matrices as parameters to loss function yielded me poor results and pytorch's loss calculation function doesn't disclose on how loss calculation is done for this case.</p></li>
  <li><p>How is the loss averaged for each instance and across the a batch?</p></li>
  <li>if loss is calculated column wise and averaged for each instance and across the batch, then how can loss be backprop'd in pytorch?</li>
  </ol>
  
  <p>Thanks in advance.</p>
  
Tags: <backpropagation><objective-functions><pytorch>
Title: Confused with backprop in pytorch with BCE loss
UserDisplayName: Stephen Philip
ThreadText: # Confused with backprop in pytorch with BCE loss
  
  (Asked by: username_0 on 2019-10-10)
  
  I've a prediction matrix(P) of dimension 3x3 and one-hot encoded label matrix(L) of dimension 3x3 as shown below.
      |0.5 0.3 0.1|      |1 0 0|
  P = |0.3 0.2 0.1|  L = |0 1 0|
      |0.2 0.5 0.8|      |0 0 1|
  
  each column in 'P' corresponds to prediction of a label in 'L'
  
  How is the BCELoss calculated using pytorch?, my experimentation by giving these two matrices as parameters to loss function yielded me poor results and pytorch's loss calculation function doesn't disclose on how loss calculation is done for this case.
  How is the loss averaged for each instance and across the a batch?
  if loss is calculated column wise and averaged for each instance and across the batch, then how can loss be backprop'd in pytorch?
  
  Thanks in advance.
  
  **Tags:** backpropagation, objective-functions, pytorch
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-10-11. Score: 1 | Accepted answer
  
  1. BCELoss ( Binary Cross Entropy Loss) is used for binary classifier, which is a neural network that have a binary output, 0 or 1. It is not used for multi-output neural network like your case. For that kind of networks, you can use MSELoss or CrossEntropyLoss as your loss for the network.
  2. For the calculation of BCE, it is shown on pytorch documentation.
     <https://pytorch.org/docs/stable/nn.html#BCELoss>
     For across batch, it either sum the loss or take the mean. you can set that through the reduction="" argument.
  3. As i said, the loss is either summed or taken the mean. Then there will be a single value so pytorch can do backdrop.
  
  Hope I can help you
  
  
  ---
  
  