Body: <p>If I understand correctly, the KL divergence is a measure of information loss between a ground truth distribution <span class="math-container">$P$</span> and a predicted distribution <span class="math-container">$Q$</span>, and the Jensen-Shannon divergence is the mean of the KL Divergences of 2 cases</p>
  
  <ol>
  <li><p>Predicted distribution is mean of <span class="math-container">$P$</span> and <span class="math-container">$Q$</span>, and ground truth is <span class="math-container">$P$</span></p></li>
  <li><p>Predicted distribution is mean of <span class="math-container">$P$</span> and <span class="math-container">$Q$</span>, and ground truth is <span class="math-container">$Q$</span> </p></li>
  </ol>
  
  <p>Since KL divergence can be easily interpreted as information loss in <span class="math-container">$Q$</span> relative to <span class="math-container">$P$</span>, what can JS divergence interpret-ably represent? I cannot see any use cases of these measures unless there are two distributions to compare. Is there any other problem where I could use them as loss functions other than generation problems? If so how, and why?  </p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-24T13:26:03.927Z
Comments.Text: this is a duplicate of https://stats.stackexchange.com/questions/117225/jensen-shannon-divergence-vs-kullback-leibler-divergence, if you look there i hope it helps
Comments.UserDisplayName: mshlis
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-24T21:51:17.837Z
Comments.Text: @mshlis Why should this be a duplicate of that?
Comments.UserDisplayName: nbro
Tags: <deep-learning><optimization><objective-functions><generative-adversarial-networks>
Title: Could the Jensen-Shannon divergence and Kullback-Leibler divergence be used as loss functions of non-generation problems?
UserDisplayName: ashenoy
ThreadText: # Could the Jensen-Shannon divergence and Kullback-Leibler divergence be used as loss functions of non-generation problems?
  
  (Asked by: username_0 on 2019-07-24)
  
  If I understand correctly, the KL divergence is a measure of information loss between a ground truth distribution $P$ and a predicted distribution $Q$, and the Jensen-Shannon divergence is the mean of the KL Divergences of 2 cases
  
  1. Predicted distribution is mean of $P$ and $Q$, and ground truth is $P$
  2. Predicted distribution is mean of $P$ and $Q$, and ground truth is $Q$
  
  Since KL divergence can be easily interpreted as information loss in $Q$ relative to $P$, what can JS divergence interpret-ably represent? I cannot see any use cases of these measures unless there are two distributions to compare. Is there any other problem where I could use them as loss functions other than generation problems? If so how, and why?
  
  **Tags:** deep-learning, optimization, objective-functions, generative-adversarial-networks
  > **Comments:**
  > **username_1**: this is a duplicate of https://stats.stackexchange.com/questions/117225/jensen-shannon-divergence-vs-kullback-leibler-divergence, if you look there i hope it helps  
  > **username_2**: @mshlis Why should this be a duplicate of that?  
  
  ---
  
  ## 0 answers
  
  