Answers.Body: <p>I understand that you want to know about methods that we can use to evaluate GANs (Generative Adversarial Networks).</p>
  
  <h1>How can GANs be evaluated?</h1>
  
  <h2>One Discriminator on Separate GANs</h2>
  
  <p>We can train a <em>Discriminator</em> beforehand and then we can use this <em>Discriminator</em> on various <em>Generators</em> to see what does this <em>Discriminator</em> say about the images generated from each <em>Generator</em>. The average output of the <em>Discriminator</em> for images generated from one <em>Generator</em> can be compared with that of the images generated from another <em>Generator</em>. This means that according to this <em>Discriminator</em>, one <em>Generator</em> is better than another.</p>
  
  <h2>Comparing Probabilistic Models</h2>
  
  <p>This is the class of evaluation metrics that is attracting research attention recently. Basically, all generative models are probabilistic in nature, even GANs. When we ask a generative model to generate something like an image, we are simply sampling from a probability distribution. This means that if we can compare the probability distributions of our <em>Generator</em> and that of our original data then we will have an evaluation metric. We can come up with a kernel function to define these probability distributions more accurately. Then we can take our generated samples from the <em>Generator</em> and see the probability of that sample being taken from the original distribution. Researchers have also utilized <em>KL-divergence</em> for comparing samples generated from two distributions.</p>
  
  <p><em>Note: KL-divergence can not be directly applied in this case because corresponding ground truth is not available. However, there are some modifications proposed in the research work mentioned.</em></p>
  
  <h1>Recommended Readings</h1>
  
  <ul>
  <li><a href="https://arxiv.org/pdf/1511.01844.pdf" rel="nofollow noreferrer">A NOTE ON THE EVALUATION OF GENERATIVE MODELS</a></li>
  <li><a href="http://www.iro.umontreal.ca/~lisa/publications/unlearning_for_better_mixing.pdf" rel="nofollow noreferrer">UNLEARNING FOR BETTER MIXING</a></li>
  <li><a href="https://openreview.net/pdf?id=SJgabgBFl" rel="nofollow noreferrer">A QUANTITATIVE MEASURE OF GENERATIVE ADVERSARIAL NETWORK DISTRIBUTIONS</a></li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-03-08T20:13:48.133Z
Answers.LastActivityDate: 2018-03-08T20:33:55.370Z
Answers.LastEditDate: 2018-03-08T20:33:55.370Z
Answers.UserDisplayName: Mohsin
Body: <p>As we all know, there has been tons of GAN variants featuring different aspects of the image generation task such as stability, resolution or the ability to manipulate images. However, it is still confusing to me that how do we determine that images generated by one network are more plausible than images generated by another?</p>
  
  <p>PS: could someone with higher reputation create more tags like image generation?</p>
  
Tags: <deep-learning><generative-model>
Title: How to evaluate the goodness of images generated by GANs?
UserDisplayName: AkiyamaYukari
ThreadText: # How to evaluate the goodness of images generated by GANs?
  
  (Asked by: username_0 on 2018-03-07)
  
  As we all know, there has been tons of GAN variants featuring different aspects of the image generation task such as stability, resolution or the ability to manipulate images. However, it is still confusing to me that how do we determine that images generated by one network are more plausible than images generated by another?
  
  PS: could someone with higher reputation create more tags like image generation?
  
  **Tags:** deep-learning, generative-model
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-03-08. Score: 1 | Accepted answer
  
  I understand that you want to know about methods that we can use to evaluate GANs (Generative Adversarial Networks).
  
  # How can GANs be evaluated?
  
  ## One Discriminator on Separate GANs
  
  We can train a *Discriminator* beforehand and then we can use this *Discriminator* on various *Generators* to see what does this *Discriminator* say about the images generated from each *Generator*. The average output of the *Discriminator* for images generated from one *Generator* can be compared with that of the images generated from another *Generator*. This means that according to this *Discriminator*, one *Generator* is better than another.
  
  ## Comparing Probabilistic Models
  
  This is the class of evaluation metrics that is attracting research attention recently. Basically, all generative models are probabilistic in nature, even GANs. When we ask a generative model to generate something like an image, we are simply sampling from a probability distribution. This means that if we can compare the probability distributions of our *Generator* and that of our original data then we will have an evaluation metric. We can come up with a kernel function to define these probability distributions more accurately. Then we can take our generated samples from the *Generator* and see the probability of that sample being taken from the original distribution. Researchers have also utilized *KL-divergence* for comparing samples generated from two distributions.
  
  *Note: KL-divergence can not be directly applied in this case because corresponding ground truth is not available. However, there are some modifications proposed in the research work mentioned.*
  
  # Recommended Readings
  
  * [A NOTE ON THE EVALUATION OF GENERATIVE MODELS](https://arxiv.org/pdf/1511.01844.pdf)
  * [UNLEARNING FOR BETTER MIXING](http://www.iro.umontreal.ca/~lisa/publications/unlearning_for_better_mixing.pdf)
  * [A QUANTITATIVE MEASURE OF GENERATIVE ADVERSARIAL NETWORK DISTRIBUTIONS](https://openreview.net/pdf?id=SJgabgBFl)
  
  
  ---
  
  