Answers.Body: <p>The answer to your question can be found in [1, sec. 4.4]. Briefly, the GAN optimization problem is a mini-max game, and early on the proposition of GANs, the authors had the idea that one should balance the power/optimization of these two actors. Here is a quote from the said reference:</p>
  <blockquote>
  <p>The author’s present belief is that GANs work by estimating the ratio of
  the data density and model density. This ratio is estimated correctly only when the discriminator is optimal, so it is fine for the discriminator to overpower the generator.</p>
  </blockquote>
  <p>It appears that, despite their initial claims in [2], the authors do not believe that this is the case anymore. In [1], Goodfellow explains,</p>
  <blockquote>
  <p>The idea that the discriminator should always be optimal in order to best
  estimate the ratio would suggest training the discriminator for k &gt; 1 steps
  every time the generator is trained for one step. In practice, this does not
  usually result in a clear improvement.</p>
  </blockquote>
  <p><strong>To summarize:</strong> when the discriminator is near optimality, the ratio between densities (<span class="math-container">$p_{data}$</span> and <span class="math-container">$p_{model}$</span>) is more accurate, generating better gradients to update <span class="math-container">$p_{model}$</span> a.k.a. the generator. If you choose to do the other way around, you would be updating the generator successively with poor gradients.</p>
  <h2>References</h2>
  <p>[1] Goodfellow, I. (2016). <a href="https://arxiv.org/pdf/1701.00160.pdf" rel="nofollow noreferrer">Nips 2016 tutorial: Generative adversarial networks</a>. arXiv preprint arXiv:1701.00160.</p>
  <p>[2] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., ... &amp; Bengio, Y. (2014). Generative adversarial nets. Advances in neural information processing systems, 27.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-03T12:46:18.443Z
Answers.LastActivityDate: 2023-09-11T11:56:50.393Z
Answers.LastEditDate: 2023-09-11T11:56:50.393Z
Answers.UserDisplayName: Eduardo Montesuma
Body: <p>In this paper <a href="https://arxiv.org/abs/1406.2661" rel="nofollow noreferrer">https://arxiv.org/abs/1406.2661</a> , the codes for training a gan are :</p>
  <p><a href="https://i.sstatic.net/Sozlv.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Sozlv.png" alt="enter image description here" /></a></p>
  <p>Why do we train the discriminator for <span class="math-container">$k$</span> steps while the generator only for <span class="math-container">$1$</span> step? Why not the other way around?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-06-03T12:43:22.873Z
Comments.Text: The discriminator is composing a nonlinear measure of goodness approximation, an advanced version of the loss.  If it gets pathological, then the generator is worse off with the GAN-wrapper to train it.  Smart folks wire a training run of the discriminator so that each training iteration, half the information through the discriminator gets is propagated through the generator in the form of training.
Comments.UserDisplayName: EngrStudent
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-09-11T12:40:09.327Z
Comments.Text: There is a very similar question (with answer) [here](https://ai.stackexchange.com/questions/41185/why-does-the-critic-of-wgan-gp-run-more-steps-than-the-generator/41192#41192)
Comments.UserDisplayName: Robin van Hoorn
Tags: <deep-learning><training><generative-adversarial-networks><generator><discriminator>
Title: Why do we train the discriminators k times but train the generator only 1 time in a iteration in GAN?
UserDisplayName: wit
ThreadText: # Why do we train the discriminators k times but train the generator only 1 time in a iteration in GAN?
  
  (Asked by: username_0 on 2022-06-03)
  
  In this paper <https://arxiv.org/abs/1406.2661> , the codes for training a gan are :
  
  [![enter image description here](https://i.sstatic.net/Sozlv.png)](https://i.sstatic.net/Sozlv.png)
  
  Why do we train the discriminator for $k$ steps while the generator only for $1$ step? Why not the other way around?
  
  **Tags:** deep-learning, training, generative-adversarial-networks, generator, discriminator
  
  > **Comments:**
  > **username_1**: The discriminator is composing a nonlinear measure of goodness approximation, an advanced version of the loss.  If it gets pathological, then the generator is worse off with the GAN-wrapper to train it.  Smart folks wire a training run of the discriminator so that each training iteration, half the information through the discriminator gets is propagated through the generator in the form of training.  
  > **username_2**: There is a very similar question (with answer) [here](https://ai.stackexchange.com/questions/41185/why-does-the-critic-of-wgan-gp-run-more-steps-than-the-generator/41192#41192)  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2022-06-03. Score: 3 | Accepted answer
  
  The answer to your question can be found in [1, sec. 4.4]. Briefly, the GAN optimization problem is a mini-max game, and early on the proposition of GANs, the authors had the idea that one should balance the power/optimization of these two actors. Here is a quote from the said reference:
  
  > The author’s present belief is that GANs work by estimating the ratio of
  > the data density and model density. This ratio is estimated correctly only when the discriminator is optimal, so it is fine for the discriminator to overpower the generator.
  
  It appears that, despite their initial claims in [2], the authors do not believe that this is the case anymore. In [1], Goodfellow explains,
  
  > The idea that the discriminator should always be optimal in order to best
  > estimate the ratio would suggest training the discriminator for k > 1 steps
  > every time the generator is trained for one step. In practice, this does not
  > usually result in a clear improvement.
  
  **To summarize:** when the discriminator is near optimality, the ratio between densities ($p\_{data}$ and $p\_{model}$) is more accurate, generating better gradients to update $p\_{model}$ a.k.a. the generator. If you choose to do the other way around, you would be updating the generator successively with poor gradients.
  
  ## References
  
  [1] Goodfellow, I. (2016). [Nips 2016 tutorial: Generative adversarial networks](https://arxiv.org/pdf/1701.00160.pdf). arXiv preprint arXiv:1701.00160.
  
  [2] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., ... & Bengio, Y. (2014). Generative adversarial nets. Advances in neural information processing systems, 27.
  
  
  ---
  
  