Answers.Body: <p>It should remain from a general code that has been refactored. By the way, the red code phrase is always zero. Because, <code>beta</code> is a vector of <code>1</code>, and <span class="math-container">$\log(\Gamma(1)) = \log(1) = 0$</span>, i.e., <code>tf.math.lgamma(beta)</code>. So, sum of zeros will be zero.</p>
  <p>As you said, the other parts of the code are clear and completely matched with the definition.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-08-08T08:07:17.913Z
Answers.Text: Thank you, you are right. I was worried because I found this code multiple times on the internet, but it seems everyone copied from the same source.
Answers.UserDisplayName: Baka
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-08-07T23:32:56.870Z
Answers.LastActivityDate: 2021-08-07T23:32:56.870Z
Answers.UserDisplayName: OmG
Body: <p>I am trying to understand a certain KL-divergence formula (which can be found on page 6 of the paper <a href="https://papers.nips.cc/paper/2018/file/a981f2b708044d6fb4a71a1463242520-Paper.pdf" rel="nofollow noreferrer">Evidential Deep Learning to Quantify Classification Uncertainty</a>) and found <a href="https://github.com/aamini/evidential-deep-learning/blob/main/evidential_deep_learning/losses/discrete.py" rel="nofollow noreferrer">a TensorFlow implementation for it</a>. I understand most parts of the formula and put colored frames around them. Unfortunately, there is one term in the implementation (underlined red) that I can't tell how it fits in the formula.</p>
  <p><a href="https://i.sstatic.net/jcFZA.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/jcFZA.png" alt="tensorflow implementation of KL divergence" /></a></p>
  <p>Is this a mistake in the implementation? I don't understand how the red part is necessary.</p>
  
Tags: <deep-learning><tensorflow><papers><implementation><kl-divergence>
Title: How is this statement from a TensorFlow implementation of a certain KL-divergence formula related to the corresponding formula?
UserDisplayName: Baka
ThreadText: # How is this statement from a TensorFlow implementation of a certain KL-divergence formula related to the corresponding formula?
  
  (Asked by: username_0 on 2021-08-07)
  
  I am trying to understand a certain KL-divergence formula (which can be found on page 6 of the paper [Evidential Deep Learning to Quantify Classification Uncertainty](https://papers.nips.cc/paper/2018/file/a981f2b708044d6fb4a71a1463242520-Paper.pdf)) and found [a TensorFlow implementation for it](https://github.com/aamini/evidential-deep-learning/blob/main/evidential_deep_learning/losses/discrete.py). I understand most parts of the formula and put colored frames around them. Unfortunately, there is one term in the implementation (underlined red) that I can't tell how it fits in the formula.
  
  [![tensorflow implementation of KL divergence](https://i.sstatic.net/jcFZA.png)](https://i.sstatic.net/jcFZA.png)
  
  Is this a mistake in the implementation? I don't understand how the red part is necessary.
  
  **Tags:** deep-learning, tensorflow, papers, implementation, kl-divergence
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-08-07. Score: 1 | Accepted answer
  
  It should remain from a general code that has been refactored. By the way, the red code phrase is always zero. Because, `beta` is a vector of `1`, and $\log(\Gamma(1)) = \log(1) = 0$, i.e., `tf.math.lgamma(beta)`. So, sum of zeros will be zero.
  
  As you said, the other parts of the code are clear and completely matched with the definition.
  
  > **Comments:**
  > **username_0**: Thank you, you are right. I was worried because I found this code multiple times on the internet, but it seems everyone copied from the same source.  
  
  ---
  
  