Answers.Body: <p>Binary cross-entropy is <em>not</em> commutative, and the two elements typically used with it in machine learning - ground truth <span class="math-container">$y$</span> and estimate of class probability <span class="math-container">$\hat{y}$</span> - are not interchangeable conceptually either.</p>
  <p>The BCE loss function is:</p>
  <p><span class="math-container">$$\mathcal{L}(y,\hat{y}) = -y\text{log}(\hat{y}) - (1-y)\text{log}(1-\hat{y})$$</span></p>
  <p>Ground truth <span class="math-container">$y$</span> is commonly either <span class="math-container">$0$</span> or <span class="math-container">$1$</span> for data records where an individual example is either a class member or not. If that value was used in place of <span class="math-container">$\hat{y}$</span> then one of logs would be <span class="math-container">$\text{log}(0)$</span> which is undefined.</p>
  <p>You can generate example outputs when both <span class="math-container">$y$</span> and <span class="math-container">$\hat{y}$</span> are between <span class="math-container">$0$</span> and <span class="math-container">$1$</span>, and see that the loss function is not symmetric:</p>
  <ul>
  <li><span class="math-container">$\mathcal{L}(0.5,0.9) \approx 1.204$</span></li>
  <li><span class="math-container">$\mathcal{L}(0.9,0.5) \approx 0.693$</span></li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-04T06:28:53.843Z
Answers.Text: We can also generate the graphic. The contour shows the asymmetry better than the 3D plot. https://www.wolframalpha.com/input?i=z%3Dx*log%28y%29%2B%281-x%29log%281-y%29
Answers.UserDisplayName: Jaume Oliver Lafont
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-03T11:42:43.057Z
Answers.LastActivityDate: 2023-11-03T13:20:03.857Z
Answers.LastEditDate: 2023-11-03T13:20:03.857Z
Answers.UserDisplayName: Neil Slater
Body: <p>Is binary cross entropy commutative? Both in math theory and in PyTorch?</p>
  <p>I'm not fresh enough on my math to tell straight by looking at the formula and code.</p>
  <p><code>BCE(x,y) =?= BCE(y,x)</code></p>
  
Tags: <binary-crossentropy>
Title: Is BCE commutative?
UserDisplayName: Thomas Eding
ThreadText: # Is BCE commutative?
  
  (Asked by: username_0 on 2023-11-03)
  
  Is binary cross entropy commutative? Both in math theory and in PyTorch?
  
  I'm not fresh enough on my math to tell straight by looking at the formula and code.
  
  `BCE(x,y) =?= BCE(y,x)`
  
  **Tags:** binary-crossentropy
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-11-03. Score: 5 | Accepted answer
  
  Binary cross-entropy is *not* commutative, and the two elements typically used with it in machine learning - ground truth $y$ and estimate of class probability $\hat{y}$ - are not interchangeable conceptually either.
  
  The BCE loss function is:
  
  $$\mathcal{L}(y,\hat{y}) = -y\text{log}(\hat{y}) - (1-y)\text{log}(1-\hat{y})$$
  
  Ground truth $y$ is commonly either $0$ or $1$ for data records where an individual example is either a class member or not. If that value was used in place of $\hat{y}$ then one of logs would be $\text{log}(0)$ which is undefined.
  
  You can generate example outputs when both $y$ and $\hat{y}$ are between $0$ and $1$, and see that the loss function is not symmetric:
  
  * $\mathcal{L}(0.5,0.9) \approx 1.204$
  * $\mathcal{L}(0.9,0.5) \approx 0.693$
  
  > **Comments:**
  > **username_2**: We can also generate the graphic. The contour shows the asymmetry better than the 3D plot. https://www.wolframalpha.com/input?i=z%3Dx*log%28y%29%2B%281-x%29log%281-y%29  
  
  ---
  
  