Answers.Body: <h1>Compare generated and real data</h1>
  
  <p>All the results produced by G are always considered "wrong" by definition, even for a very good generator.</p>
  
  <p>You provide the discriminative neural network <span class="math-container">$D$</span> with a mix of results generated by the generator network <span class="math-container">$G$</span> and real results from an outside source, and then you train it to distinguish if the result was produced by the generator or not - you're not comparing "good" and "bad" results, you're comparing real versus generated results.</p>
  
  <p>This will result in a "mutual evolution" as <span class="math-container">$D$</span> will learn to find features that separate real results from generated ones, and <span class="math-container">$G$</span> will learn how to generate results that are hard to distinguish from real data.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2016-10-24T19:20:48.323Z
Answers.LastActivityDate: 2019-04-15T16:51:11.860Z
Answers.LastEditDate: 2019-04-15T16:51:11.860Z
Answers.UserDisplayName: Peteris
Answers.Body: <p>A <em>discriminative</em> network (<span class="math-container">$D$</span>) <strong>learns</strong> to <em>discriminate</em> by definition - we provide it with the true and the generated data, and let it learn by itself how to discriminate between the two. </p>
  
  <p>Therefore, we expect network <span class="math-container">$D$</span> to improve the ability of network <span class="math-container">$G$</span> to generate better and better images (or other kind of data), as it try to "trick" network <span class="math-container">$D$</span> by producing new data that is more similar to "real data". It is not about the accuracy of network <span class="math-container">$D$</span> at all. <strong>It is not about improving the accuracy</strong>, it is about improving the ability of the computer to generate more "believable" data.</p>
  
  <p>That said, using this scenario could be a good "unsupervised" way to improve the classification power of neural networks, as it forces the generator model to learn better features of real data, and to learn how to distinguish between actual features and noise, using much less data that is needed for a traditional supervised learning scheme. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2016-10-25T11:58:19.457Z
Answers.LastActivityDate: 2019-04-15T16:18:28.033Z
Answers.LastEditDate: 2019-04-15T16:18:28.033Z
Answers.UserDisplayName: Dror Hilman
Body: <p>I am reading about generative adversarial networks (GANs) and I have some doubts regarding it. So far, I understand that in a GAN there are two different types of neural networks: one is generative (<span class="math-container">$G$</span>) and the other discriminative (<span class="math-container">$D$</span>). The generative neural network generates some data which the discriminative neural network judges for correctness.</p>
  <p>How do the discriminative (<span class="math-container">$D$</span>) neural nets initially know whether the data produced by <span class="math-container">$G$</span> is correct or not? Do we have to train the <span class="math-container">$D$</span> first then add it into the GAN with <span class="math-container">$G$</span>?</p>
  <p>Let's consider my trained <span class="math-container">$D$</span> net, which can classify a picture with 90% accuracy. If we add this <span class="math-container">$D$</span> to a GAN, there is a 10% probability it will classify an image wrong. If we train a GAN with this <span class="math-container">$D$</span>, then will it also have the same 10% error in classifying an image? If yes, then why do GANs show promising results?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-11-22T12:32:35.420Z
Comments.Text: I changed the title because your question is more about "how are GANs trained" rather than "how do they work" or "what are they". I suggest that you review the answers below and accept of one those, if they are good enough.
Comments.UserDisplayName: nbro
Tags: <neural-networks><deep-learning><training><generative-adversarial-networks><self-supervised-learning>
Title: How are generative adversarial networks trained?
UserDisplayName: Eka
ThreadText: # How are generative adversarial networks trained?
  
  (Asked by: username_0 on 2016-10-24)
  
  I am reading about generative adversarial networks (GANs) and I have some doubts regarding it. So far, I understand that in a GAN there are two different types of neural networks: one is generative ($G$) and the other discriminative ($D$). The generative neural network generates some data which the discriminative neural network judges for correctness.
  
  How do the discriminative ($D$) neural nets initially know whether the data produced by $G$ is correct or not? Do we have to train the $D$ first then add it into the GAN with $G$?
  
  Let's consider my trained $D$ net, which can classify a picture with 90% accuracy. If we add this $D$ to a GAN, there is a 10% probability it will classify an image wrong. If we train a GAN with this $D$, then will it also have the same 10% error in classifying an image? If yes, then why do GANs show promising results?
  
  **Tags:** neural-networks, deep-learning, training, generative-adversarial-networks, self-supervised-learning
  
  > **Comments:**
  > **username_1**: I changed the title because your question is more about "how are GANs trained" rather than "how do they work" or "what are they". I suggest that you review the answers below and accept of one those, if they are good enough.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2016-10-24. Score: 4
  
  # Compare generated and real data
  
  All the results produced by G are always considered "wrong" by definition, even for a very good generator.
  
  You provide the discriminative neural network $D$ with a mix of results generated by the generator network $G$ and real results from an outside source, and then you train it to distinguish if the result was produced by the generator or not - you're not comparing "good" and "bad" results, you're comparing real versus generated results.
  
  This will result in a "mutual evolution" as $D$ will learn to find features that separate real results from generated ones, and $G$ will learn how to generate results that are hard to distinguish from real data.
  
  
  ---
  
  ### Answer by username_3 on 2016-10-25. Score: 2
  
  A *discriminative* network ($D$) **learns** to *discriminate* by definition - we provide it with the true and the generated data, and let it learn by itself how to discriminate between the two.
  
  Therefore, we expect network $D$ to improve the ability of network $G$ to generate better and better images (or other kind of data), as it try to "trick" network $D$ by producing new data that is more similar to "real data". It is not about the accuracy of network $D$ at all. **It is not about improving the accuracy**, it is about improving the ability of the computer to generate more "believable" data.
  
  That said, using this scenario could be a good "unsupervised" way to improve the classification power of neural networks, as it forces the generator model to learn better features of real data, and to learn how to distinguish between actual features and noise, using much less data that is needed for a traditional supervised learning scheme.
  
  
  ---
  
  