Answers.Body: <p>After further research, I have found the answer. nbro was of course right, the weighting is not implementation dependant, it was already introduced in the paper (<a href="https://arxiv.org/pdf/1611.07004.pdf" rel="nofollow noreferrer">arXiv</a>). However, there is minimal information about that, only it is mentioned within the optimization function:</p>
  <p><span class="math-container">$\arg \min_G \max_D \mathcal{L}_{cGAN}(G,D) + \lambda \mathcal{L}_{L1}(G)$</span></p>
  <p>In fact, the parameter lambda stands for weight of <span class="math-container">$L_1$</span> loss. There should be a second parameter <span class="math-container">$\epsilon$</span> for <span class="math-container">$\mathcal{L}_{cGAN}(G,D)$</span> because in real implementation you can actually change that as well. In theory, however, only modulation of <span class="math-container">$L_1$</span> loss is needed. The paper does not state why the <span class="math-container">$\lambda$</span> parameter is needed, only mentions that setting it to 0 will lead to pure cGAN implementation. <a href="https://machinelearningmastery.com/a-gentle-introduction-to-pix2pix-generative-adversarial-network/" rel="nofollow noreferrer">David Brownlee in his blog</a> states:</p>
  <blockquote>
  <p>The adversarial loss influences whether the generator model can output
  images that are plausible in the target domain, whereas the L1 loss
  regularizes the generator model to output images that are a plausible
  translation of the source image. As such, the combination of the L1
  loss to the adversarial loss is controlled by a new hyperparameter
  lambda, which is set to 10, e.g. giving 10 times the importance of the
  L1 loss than the adversarial loss to the generator during training.</p>
  </blockquote>
  <p>The loss weights thus are hyperparameters that tell the network how much plausible translation of the source image do we need.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-27T19:05:29.943Z
Answers.Text: In addition, I would like to point out that I've seen this weighting of one component of the loss also being used in the context of _variational auto-encoders (VAEs)_ and _variational Bayesian neural networks_, where the loss function is usually the [ELBO (Evidence Lower BOund)](https://ai.stackexchange.com/a/17900/2444), which is composed of 2 terms, which can be weighted. There's actually a little bit of debate about what the correct weighting should be (at least in the context of variational BNNs).
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-27T19:00:31.213Z
Answers.LastActivityDate: 2021-01-27T19:00:31.213Z
Answers.UserDisplayName: Ruli
Body: <p>I am working on a <a href="https://arxiv.org/abs/1611.07004" rel="nofollow noreferrer">pix2pix GAN model</a> that was inspired by the code in this <a href="https://github.com/eriklindernoren/Keras-GAN/blob/master/pix2pix/pix2pix.py" rel="nofollow noreferrer">Github repository</a>. The original code is working and I have already customized most of the code for my needs. However, there is one part I am unable to understand.</p>
  <p>The pix2pix GAN is a conditional GAN network that takes an image as a condition and outputs a modified image - such as blurry to clear, facades to buildings, filling up cut out part of an image, etc. The combined model thus takes as input a conditional image, the discriminator compares it with the dummy matrix named valid or fake, containing 0s or 1s according to validity (0 for generated samples, 1 for real samples). The generator loss is according to similarity with real sample + discriminator. The following code corresponds to what I told:</p>
  <pre><code>self.combined = Model(inputs=[img_A, img_B], outputs=[valid, fake_A])
  self.combined.compile(loss=['mse', 'mae'],
                        loss_weights=[1, 100],
                        optimizer=optimizer)
  </code></pre>
  <p>The losses are thus set as MSE for discriminator output and MAE for generator. That seems to be OK, but I can not understand why the implementation uses 1 and 100 for the weights of the discriminator and generator losses, respectively, which seems to imply that the discriminator loss is 100 times lower than the loss of the generator. I couldn't find the reason in the original article. Are my understandings of the GAN incorrect?</p>
  <p>Disclaimer: I have posted this question on <a href="https://stats.stackexchange.com/q/504673/82135">Stats SE</a>, but have no luck with answers. Maybe it is more suitable for AI.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-19T11:48:06.510Z
Comments.Text: Maybe you can take a look at the original/official implementation of pix2pix and see if they are doing something similar: https://github.com/phillipi/pix2pix.
Comments.UserDisplayName: nbro
Tags: <objective-functions><generative-adversarial-networks>
Title: In this implementation of pix2pix, why are the weights for the discriminator and generator losses set to 1 and 100 respectively?
UserDisplayName: Ruli
ThreadText: # In this implementation of pix2pix, why are the weights for the discriminator and generator losses set to 1 and 100 respectively?
  
  (Asked by: username_0 on 2021-01-18)
  
  I am working on a [pix2pix GAN model](https://arxiv.org/abs/1611.07004) that was inspired by the code in this [Github repository](https://github.com/eriklindernoren/Keras-GAN/blob/master/pix2pix/pix2pix.py). The original code is working and I have already customized most of the code for my needs. However, there is one part I am unable to understand.
  
  The pix2pix GAN is a conditional GAN network that takes an image as a condition and outputs a modified image - such as blurry to clear, facades to buildings, filling up cut out part of an image, etc. The combined model thus takes as input a conditional image, the discriminator compares it with the dummy matrix named valid or fake, containing 0s or 1s according to validity (0 for generated samples, 1 for real samples). The generator loss is according to similarity with real sample + discriminator. The following code corresponds to what I told:
  
  ```
  self.combined = Model(inputs=[img_A, img_B], outputs=[valid, fake_A])
  self.combined.compile(loss=['mse', 'mae'],
                        loss_weights=[1, 100],
                        optimizer=optimizer)
  
  ```
  
  The losses are thus set as MSE for discriminator output and MAE for generator. That seems to be OK, but I can not understand why the implementation uses 1 and 100 for the weights of the discriminator and generator losses, respectively, which seems to imply that the discriminator loss is 100 times lower than the loss of the generator. I couldn't find the reason in the original article. Are my understandings of the GAN incorrect?
  
  Disclaimer: I have posted this question on [Stats SE](https://stats.stackexchange.com/q/504673/82135), but have no luck with answers. Maybe it is more suitable for AI.
  
  **Tags:** objective-functions, generative-adversarial-networks
  
  > **Comments:**
  > **username_1**: Maybe you can take a look at the original/official implementation of pix2pix and see if they are doing something similar: https://github.com/phillipi/pix2pix.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2021-01-27. Score: 1 | Accepted answer
  
  After further research, I have found the answer. username_1 was of course right, the weighting is not implementation dependant, it was already introduced in the paper ([arXiv](https://arxiv.org/pdf/1611.07004.pdf)). However, there is minimal information about that, only it is mentioned within the optimization function:
  
  $\arg \min\_G \max\_D \mathcal{L}\_{cGAN}(G,D) + \lambda \mathcal{L}\_{L1}(G)$
  
  In fact, the parameter lambda stands for weight of $L\_1$ loss. There should be a second parameter $\epsilon$ for $\mathcal{L}\_{cGAN}(G,D)$ because in real implementation you can actually change that as well. In theory, however, only modulation of $L\_1$ loss is needed. The paper does not state why the $\lambda$ parameter is needed, only mentions that setting it to 0 will lead to pure cGAN implementation. [David Brownlee in his blog](https://machinelearningmastery.com/a-gentle-introduction-to-pix2pix-generative-adversarial-network/) states:
  
  > The adversarial loss influences whether the generator model can output
  > images that are plausible in the target domain, whereas the L1 loss
  > regularizes the generator model to output images that are a plausible
  > translation of the source image. As such, the combination of the L1
  > loss to the adversarial loss is controlled by a new hyperparameter
  > lambda, which is set to 10, e.g. giving 10 times the importance of the
  > L1 loss than the adversarial loss to the generator during training.
  
  The loss weights thus are hyperparameters that tell the network how much plausible translation of the source image do we need.
  
  > **Comments:**
  > **username_1**: In addition, I would like to point out that I've seen this weighting of one component of the loss also being used in the context of _variational auto-encoders (VAEs)_ and _variational Bayesian neural networks_, where the loss function is usually the [ELBO (Evidence Lower BOund)](https://ai.stackexchange.com/a/17900/2444), which is composed of 2 terms, which can be weighted. There's actually a little bit of debate about what the correct weighting should be (at least in the context of variational BNNs).  
  
  ---
  
  