Body: <p>In the training of a Generative Adversarial Networks (GAN) system, a perfect discriminator (D) is one which outputs 1 (&quot;true image&quot;) for all images of the training dataset and 0 (&quot;false image&quot;) for all images created by the generator (G).</p>
  <p>I've read on separate occasions that -when trained with the original GAN framework and loss function as described in the Goodfellow 2014 paper- if this &quot;perfect D state&quot; occurs during training then this is a &quot;failure mode&quot;, sometimes refered to as &quot;<strong>vanishing gradients</strong>&quot; or &quot;convergence failure&quot;, from which G &quot;cannot recover&quot;.</p>
  <p>For instance :</p>
  <blockquote>
  <p>&quot;When the discriminator is perfect (...), the loss function falls to zero and we end up with no gradient to update the loss during learning iterations&quot; (<a href="https://lilianweng.github.io/posts/2017-08-20-gan/" rel="nofollow noreferrer">source</a>, well-recieved blog post)</p>
  </blockquote>
  <blockquote>
  <p>&quot;An optimal discriminator doesn't provide enough information for the generator to make progress.&quot; (<a href="https://developers.google.com/machine-learning/gan/problems?hl=en" rel="nofollow noreferrer">source</a>, Google developers)</p>
  </blockquote>
  <blockquote>
  <p>[T]he generator score approaches zero and does not recover. (...)
  In this case, the discriminator classifies most of the images
  correctly. In turn, the generator cannot produce any images that fool
  the discriminator and thus fails to learn. (<a href="https://www.mathworks.com/help/deeplearning/ug/monitor-gan-training-progress-and-identify-common-failure-modes.html" rel="nofollow noreferrer">source</a>, Mathworks)</p>
  </blockquote>
  <p>The <a href="https://arxiv.org/pdf/1701.04862.pdf" rel="nofollow noreferrer">Towards Principled Methods for training GANs</a> paper (Arjovsky &amp; Bottou, 2017) is perhaps a more accurate source on the matter as it dives deeper in the theory by proving that since the loss function relates to the Jensen-Shannon distance, it saturates when the distributions of real and generated images are disjoint (hence no gradient).</p>
  <blockquote>
  <p>If the two distributions we care about have supports that are disjoint
  or lie on low dimensional manifolds, the optimal discriminator will be
  perfect and its gradient will be zero almost everywhere. (section 2.2)</p>
  </blockquote>
  <p>I understand the gist of the theory developed in that paper, but the result seems <strong>nevertheless counter-intuitive to me</strong> when I try to think about them in the context of the GAN training procedure. Indeed, my undertanding of a GAN training iteration is the following :</p>
  <ol>
  <li>Show true images to D, train D to output 'true'.</li>
  <li>Show false (created by G) images to D, train D to output 'false'.</li>
  <li><strong>Show false images to D, train G such that D outputs 'true'.</strong></li>
  </ol>
  <p>I can see that in the case of a perfect D, steps 1 and 2 would lead to a loss of zero and hence no gradient.<br />
  But I would expect that in step 3 :</p>
  <ol>
  <li>As a perfect D would predict all images to be 'false' and the loss would be computed by comparing its answers to predicting all 'true's, we'd have a high loss.</li>
  <li>Thus, we would get a high gradient.</li>
  <li>By backprop, this gradient would lead to the identifcation of the most salient features in the images which D is using to predict them as false.</li>
  <li>This would provide valuable information to G to improve its false images to better match the training set.</li>
  </ol>
  <p>So looking at it that way, it doesn't seem to me that a perfect D should lead to &quot;vanishing gradients&quot; and G being unable to recover from it.</p>
  <p><strong>What's wrong with my understanding of the training process and why it is not compatible with the results from the Arjovsky paper ?</strong></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-02T11:16:32.927Z
Comments.Text: The source you state is slightly iffy imo. Its a paper talking about 'introducing the WGAN', even though it was introduced by Arjovsky et al. in 2017 (2 years prior). Do you have other references which state that the generator does not learn anymore afterwards?
Comments.UserDisplayName: Robin van Hoorn
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-02T12:41:20.027Z
Comments.Text: @RobinvanHoorn The original source is actually a blog post (2017) made into a paper two years later. I've re-worked the question to add more sources and highlight better the discrepancy between my understanding and the theory developed in the Arjovsky "Towards principled ..." paper.
Comments.UserDisplayName: Soltius
Tags: <training><generative-adversarial-networks><loss><gradient><wasserstein-gan>
Title: GAN : Why does a perfect discriminator mean no gradient for the generator?
UserDisplayName: Soltius
ThreadText: # GAN : Why does a perfect discriminator mean no gradient for the generator?
  
  (Asked by: username_0 on 2023-02-02)
  
  In the training of a Generative Adversarial Networks (GAN) system, a perfect discriminator (D) is one which outputs 1 ("true image") for all images of the training dataset and 0 ("false image") for all images created by the generator (G).
  
  I've read on separate occasions that -when trained with the original GAN framework and loss function as described in the Goodfellow 2014 paper- if this "perfect D state" occurs during training then this is a "failure mode", sometimes refered to as "**vanishing gradients**" or "convergence failure", from which G "cannot recover".
  
  For instance :
  
  > "When the discriminator is perfect (...), the loss function falls to zero and we end up with no gradient to update the loss during learning iterations" ([source](https://lilianweng.github.io/posts/2017-08-20-gan/), well-recieved blog post)
  
  > "An optimal discriminator doesn't provide enough information for the generator to make progress." ([source](https://developers.google.com/machine-learning/gan/problems?hl=en), Google developers)
  
  > [T]he generator score approaches zero and does not recover. (...)
  > In this case, the discriminator classifies most of the images
  > correctly. In turn, the generator cannot produce any images that fool
  > the discriminator and thus fails to learn. ([source](https://www.mathworks.com/help/deeplearning/ug/monitor-gan-training-progress-and-identify-common-failure-modes.html), Mathworks)
  
  The [Towards Principled Methods for training GANs](https://arxiv.org/pdf/1701.04862.pdf) paper (Arjovsky & Bottou, 2017) is perhaps a more accurate source on the matter as it dives deeper in the theory by proving that since the loss function relates to the Jensen-Shannon distance, it saturates when the distributions of real and generated images are disjoint (hence no gradient).
  
  > If the two distributions we care about have supports that are disjoint
  > or lie on low dimensional manifolds, the optimal discriminator will be
  > perfect and its gradient will be zero almost everywhere. (section 2.2)
  
  I understand the gist of the theory developed in that paper, but the result seems **nevertheless counter-intuitive to me** when I try to think about them in the context of the GAN training procedure. Indeed, my undertanding of a GAN training iteration is the following :
  
  1. Show true images to D, train D to output 'true'.
  2. Show false (created by G) images to D, train D to output 'false'.
  3. **Show false images to D, train G such that D outputs 'true'.**
  
  I can see that in the case of a perfect D, steps 1 and 2 would lead to a loss of zero and hence no gradient.  
  But I would expect that in step 3 :
  
  1. As a perfect D would predict all images to be 'false' and the loss would be computed by comparing its answers to predicting all 'true's, we'd have a high loss.
  2. Thus, we would get a high gradient.
  3. By backprop, this gradient would lead to the identifcation of the most salient features in the images which D is using to predict them as false.
  4. This would provide valuable information to G to improve its false images to better match the training set.
  
  So looking at it that way, it doesn't seem to me that a perfect D should lead to "vanishing gradients" and G being unable to recover from it.
  
  **What's wrong with my understanding of the training process and why it is not compatible with the results from the Arjovsky paper ?**
  
  **Tags:** training, generative-adversarial-networks, loss, gradient, wasserstein-gan
  > **Comments:**
  > **username_1**: The source you state is slightly iffy imo. Its a paper talking about 'introducing the WGAN', even though it was introduced by Arjovsky et al. in 2017 (2 years prior). Do you have other references which state that the generator does not learn anymore afterwards?  
  > **username_0**: @RobinvanHoorn The original source is actually a blog post (2017) made into a paper two years later. I've re-worked the question to add more sources and highlight better the discrepancy between my understanding and the theory developed in the Arjovsky "Towards principled ..." paper.  
  
  ---
  
  ## 0 answers
  
  