Answers.Body: <p>If you ask this question it means you conceive a generative adversarial network as a combination of 2 separate entities, the discriminator and generator, but this is not really the case.</p>
  <p>It is true that for convenience we distinguish between generator and discriminator since they fulfill separate purposes, but by simply looking at a drawing of a whole GAN you'll see that they are not separated at all.</p>
  <p>When training the generator we simply backpropagate the gradients coming from the discriminator through the fake generated sample, as it was an intermediate layer between generator and discriminator (see red area in the drawing).</p>
  <p>Of course the discriminator can be updated on its own, and even the generator since we can compute loss and backpropagate from the fake generated sample level. This is nothing special, for example it's done also in normal CNN training when using losses computed at the feature maps level (like perceptual loss).</p>
  <p><a href="https://i.sstatic.net/oJ5GS.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/oJ5GS.png" alt="enter image description here" /></a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-03-16T11:54:33.363Z
Answers.Text: Maybe reference the separate D/G training steps for detail at the end, as you cover how the architecture supports this. I suspect the OP would appreciate a reference to the full algoirithm.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-03-16T21:37:16.217Z
Answers.Text: Thanks for the answer @edoardo, but to confirm, the backprop stage goes from the discriminators output back through to its inputs (the generated image) and continues backwards though the generator? So each pixel the generator output is being corrected by the chain-rule back through the discriminator?
Answers.UserDisplayName: NeomerArcana
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-03-17T07:34:04.583Z
Answers.Text: @NeomerArcana yes, exactly! And this doesn't require any fancy implementation when using deep learning libraries, as long as both, discriminator and generator are implemented as modules, autograd  automatically recognize that the fake image has gradients coming from the generator, so when backpropagating the discriminator loss you're already updating the generator as well. The only difference required is a different optimizer for the generator weights.
Answers.UserDisplayName: Edoardo Guerriero
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-03-16T09:43:01.807Z
Answers.LastActivityDate: 2022-03-16T09:43:01.807Z
Answers.UserDisplayName: Edoardo Guerriero
Answers.Body: <p>I'm going to keep my answer relatively high-level and avoid details like the actual loss functions or activation functions. But please know these also have an effect on GANs.</p>
  <h2>The Discriminator (D)</h2>
  <p>The discriminator in a GAN is a binary classifier. It is given an image and asked to predict whether that image is <code>real</code> or <code>fake</code>. The discriminator reduces the given image down using convolutional layers. Convolutional layers detect features, in the case of images these would be edges, borders, certain shapes and so on.</p>
  <p><strong>The discriminator is learning which features correspond to a <code>real</code> image and which features correspond to a <code>fake</code> image.</strong></p>
  <h2>The Generator (G)</h2>
  <p>The generator is almost the inverse of the discriminator: Instead of reducing features it uses deconvolutional layers to create features from a random seed. This random seed is a <em>vector</em> (not a matrix) called the <em>latent vector</em> (z).</p>
  <p><strong>The generator is learning which <em>features</em> to create to return a <code>real</code> label from the discriminator.</strong></p>
  <h2>How GAN learns</h2>
  <p>In each training step the following happens:</p>
  <ol>
  <li>Get gen_images from G given z.</li>
  <li>Get real_predictions from D by passing real images to D.</li>
  <li>Get fake_predictions from D by passing gen_images to D.</li>
  <li>Compute loss on G as a function of (<code>real</code>, fake_predictions)</li>
  <li>Compute loss on D as a function of (<code>real</code>, real_predictions) + (<code>fake</code>, fake_predictions)</li>
  <li>Use backpropagation to update the weights and biases in D and G for the losses.</li>
  </ol>
  <p>The big point here is that the generator's loss function directly depends on the output of D. Did G manage to trick D or not? This is computed by comparing the <code>fake_predictions</code> with the <code>real</code> label. G wants those two values to be as close as possible.</p>
  <p>This loss does not correspond to individual pixels, but because we are using convolutions/deconvolutions in our models it corresponds to feature selection (in D) and feature creation (in G) with groups of pixels.</p>
  <p><strong>In short:</strong> the individual pixels are not <em>directly</em> trained in a GAN's generator, instead patterns and features creation are.</p>
  <p><strong>If you are still confused</strong>: Think about how a child draws.</p>
  <p>A child doesn't start drawing by examining things a millimeter at a time, but by using symbolism and feature selection. At the beginning you can really only guess what they are trying to make.</p>
  <p>A cat might be a big mess of scribbles, but slowly the child learns to draw pointy ears, or whiskers. At some point the child can draw features that are identifiable as a cat. The child has gotten feedback from you when you ask her, &quot;Oh, what's that supposed to be?&quot; and when you say, &quot;Oh, is that a cat?&quot; and then finally, &quot;That's a nice looking cat!&quot;.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-03-16T21:51:59.353Z
Answers.Text: How does this work in practice? If the discriminator predicts correctly that the image is a fake, aren't the outputs of the generator (the image pixels) corrected?
Answers.UserDisplayName: NeomerArcana
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-03-17T14:54:52.427Z
Answers.Text: Yes! That's what the loss function (for the generator) does. The loss function quantifies how _undesired_ the output was. Since G wants to trick D, we compare the guesses of D (the actual output) on the generated images and compare the guesses to the label `real` (the desired output). Like in any other network, when the desired output and the actual output don't match there is a loss, which is backpropagated through the network.
Answers.UserDisplayName: LittleLulatsch
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-03-16T13:41:04.440Z
Answers.LastActivityDate: 2022-03-16T13:41:04.440Z
Answers.UserDisplayName: LittleLulatsch
Body: <p>If the Generator in a GAN is taking a matrix of size <code>WxH</code> of noise to generate a <code>WxH</code> sized output image, and the Discriminator classifies the output as fake, how is that information back-propagated through the generator?</p>
  <p>How is the error in classification attributed to individual &quot;pixels&quot; of the generators generated image? Is the error divided by the number of pixels?</p>
  
Tags: <generative-adversarial-networks>
Title: How is the output of the Generator in a GAN corrected?
UserDisplayName: NeomerArcana
ThreadText: # How is the output of the Generator in a GAN corrected?
  
  (Asked by: username_0 on 2022-03-16)
  
  If the Generator in a GAN is taking a matrix of size `WxH` of noise to generate a `WxH` sized output image, and the Discriminator classifies the output as fake, how is that information back-propagated through the generator?
  
  How is the error in classification attributed to individual "pixels" of the generators generated image? Is the error divided by the number of pixels?
  
  **Tags:** generative-adversarial-networks
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2022-03-16. Score: 1
  
  If you ask this question it means you conceive a generative adversarial network as a combination of 2 separate entities, the discriminator and generator, but this is not really the case.
  
  It is true that for convenience we distinguish between generator and discriminator since they fulfill separate purposes, but by simply looking at a drawing of a whole GAN you'll see that they are not separated at all.
  
  When training the generator we simply backpropagate the gradients coming from the discriminator through the fake generated sample, as it was an intermediate layer between generator and discriminator (see red area in the drawing).
  
  Of course the discriminator can be updated on its own, and even the generator since we can compute loss and backpropagate from the fake generated sample level. This is nothing special, for example it's done also in normal CNN training when using losses computed at the feature maps level (like perceptual loss).
  
  [![enter image description here](https://i.sstatic.net/oJ5GS.png)](https://i.sstatic.net/oJ5GS.png)
  
  > **Comments:**
  > **username_2**: Maybe reference the separate D/G training steps for detail at the end, as you cover how the architecture supports this. I suspect the OP would appreciate a reference to the full algoirithm.  
  > **username_0**: Thanks for the answer @username_1, but to confirm, the backprop stage goes from the discriminators output back through to its inputs (the generated image) and continues backwards though the generator? So each pixel the generator output is being corrected by the chain-rule back through the discriminator?  
  > **username_1**: @username_0 yes, exactly! And this doesn't require any fancy implementation when using deep learning libraries, as long as both, discriminator and generator are implemented as modules, autograd  automatically recognize that the fake image has gradients coming from the generator, so when backpropagating the discriminator loss you're already updating the generator as well. The only difference required is a different optimizer for the generator weights.  
  
  ---
  
  ### Answer by username_3 on 2022-03-16. Score: 0
  
  I'm going to keep my answer relatively high-level and avoid details like the actual loss functions or activation functions. But please know these also have an effect on GANs.
  
  ## The Discriminator (D)
  
  The discriminator in a GAN is a binary classifier. It is given an image and asked to predict whether that image is `real` or `fake`. The discriminator reduces the given image down using convolutional layers. Convolutional layers detect features, in the case of images these would be edges, borders, certain shapes and so on.
  
  **The discriminator is learning which features correspond to a `real` image and which features correspond to a `fake` image.**
  
  ## The Generator (G)
  
  The generator is almost the inverse of the discriminator: Instead of reducing features it uses deconvolutional layers to create features from a random seed. This random seed is a *vector* (not a matrix) called the *latent vector* (z).
  
  **The generator is learning which *features* to create to return a `real` label from the discriminator.**
  
  ## How GAN learns
  
  In each training step the following happens:
  
  1. Get gen\_images from G given z.
  2. Get real\_predictions from D by passing real images to D.
  3. Get fake\_predictions from D by passing gen\_images to D.
  4. Compute loss on G as a function of (`real`, fake\_predictions)
  5. Compute loss on D as a function of (`real`, real\_predictions) + (`fake`, fake\_predictions)
  6. Use backpropagation to update the weights and biases in D and G for the losses.
  
  The big point here is that the generator's loss function directly depends on the output of D. Did G manage to trick D or not? This is computed by comparing the `fake_predictions` with the `real` label. G wants those two values to be as close as possible.
  
  This loss does not correspond to individual pixels, but because we are using convolutions/deconvolutions in our models it corresponds to feature selection (in D) and feature creation (in G) with groups of pixels.
  
  **In short:** the individual pixels are not *directly* trained in a GAN's generator, instead patterns and features creation are.
  
  **If you are still confused**: Think about how a child draws.
  
  A child doesn't start drawing by examining things a millimeter at a time, but by using symbolism and feature selection. At the beginning you can really only guess what they are trying to make.
  
  A cat might be a big mess of scribbles, but slowly the child learns to draw pointy ears, or whiskers. At some point the child can draw features that are identifiable as a cat. The child has gotten feedback from you when you ask her, "Oh, what's that supposed to be?" and when you say, "Oh, is that a cat?" and then finally, "That's a nice looking cat!".
  
  > **Comments:**
  > **username_0**: How does this work in practice? If the discriminator predicts correctly that the image is a fake, aren't the outputs of the generator (the image pixels) corrected?  
  > **username_3**: Yes! That's what the loss function (for the generator) does. The loss function quantifies how _undesired_ the output was. Since G wants to trick D, we compare the guesses of D (the actual output) on the generated images and compare the guesses to the label `real` (the desired output). Like in any other network, when the desired output and the actual output don't match there is a loss, which is backpropagated through the network.  
  
  ---
  
  