Answers.Body: <p>There are some definitions that may cause confusion here.</p>
  <p>In the original GANs (the first formula), the output from the discriminator connects to a sigmoid activation, the second formula is the real value from the last layer of the D model which you can find in section 3.1</p>
  <blockquote>
  <p>note that this convention slightly differs from the standard formulation
  in that in this case the discriminator outputs the real-valued “logits”</p>
  </blockquote>
  <p>Rewrite the first formula, consider the output from <span class="math-container">$D(x)$</span> is the real value, it will become:</p>
  <p><span class="math-container">$$ \min_{G} \max_{D} V(D, G)=\mathbb{E}_{\boldsymbol{x} \sim p_{\text {data }}(\boldsymbol{x})}[\log {\frac{1}{1+exp(-D(x))}})]+\mathbb{E}_{\boldsymbol{z} \sim p_{\boldsymbol{z}}(\boldsymbol{z})}[\log (1-\frac{1}{1+exp(-D(G(z)))})] $$</span></p>
  <p>Now, it's able to proof the first and second formula is the same.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-04-22T05:37:04.027Z
Answers.LastActivityDate: 2022-04-22T05:37:04.027Z
Answers.UserDisplayName: CuCaRot
Body: <p>I am trying to understand how the loss function <a href="https://arxiv.org/pdf/1406.2661.pdf" rel="nofollow noreferrer">from the original GAN paper</a></p>
  <p><span class="math-container">$$\min_{G} \max_{D} V(D, G)=\mathbb{E}_{\boldsymbol{x} \sim p_{\text {data }}(\boldsymbol{x})}[\log D(\boldsymbol{x})]+\mathbb{E}_{\boldsymbol{z} \sim p_{\boldsymbol{z}}(\boldsymbol{z})}[\log (1-D(G(\boldsymbol{z})))]$$</span></p>
  <p>relates to generalization in the <a href="https://arxiv.org/pdf/1706.04156.pdf" rel="nofollow noreferrer">Nagarajan &amp; Kolter (2017) </a></p>
  <p><span class="math-container">$$\min_{G} \max_{D} V(D, G)=\mathbb{E}_{\boldsymbol{x} \sim p_{\text {data }}(\boldsymbol{x})}[f(D(x))]+\mathbb{E}_{\boldsymbol{z} \sim p_{\boldsymbol{z}}(\boldsymbol{z})}[f(-D(G(z)))]$$</span></p>
  <p>where supposedly the original loss function could be recovered via</p>
  <p><span class="math-container">$$f(x) = -\log(1 + \exp(-x))$$</span></p>
  <p>But when I try to substitute <span class="math-container">$f$</span> into the generalized version of the loss function I am not getting the original one. What would be the correct steps to recover the original loss function?</p>
  
Tags: <machine-learning><papers><generative-adversarial-networks><generative-model><notation>
Title: What are the steps to derive the original GAN loss function from the generalized version?
UserDisplayName: Paul
ThreadText: # What are the steps to derive the original GAN loss function from the generalized version?
  
  (Asked by: username_0 on 2022-04-20)
  
  I am trying to understand how the loss function [from the original GAN paper](https://arxiv.org/pdf/1406.2661.pdf)
  
  $$\min\_{G} \max\_{D} V(D, G)=\mathbb{E}\_{\boldsymbol{x} \sim p\_{\text {data }}(\boldsymbol{x})}[\log D(\boldsymbol{x})]+\mathbb{E}\_{\boldsymbol{z} \sim p\_{\boldsymbol{z}}(\boldsymbol{z})}[\log (1-D(G(\boldsymbol{z})))]$$
  
  relates to generalization in the [Nagarajan & Kolter (2017)](https://arxiv.org/pdf/1706.04156.pdf)
  
  $$\min\_{G} \max\_{D} V(D, G)=\mathbb{E}\_{\boldsymbol{x} \sim p\_{\text {data }}(\boldsymbol{x})}[f(D(x))]+\mathbb{E}\_{\boldsymbol{z} \sim p\_{\boldsymbol{z}}(\boldsymbol{z})}[f(-D(G(z)))]$$
  
  where supposedly the original loss function could be recovered via
  
  $$f(x) = -\log(1 + \exp(-x))$$
  
  But when I try to substitute $f$ into the generalized version of the loss function I am not getting the original one. What would be the correct steps to recover the original loss function?
  
  **Tags:** machine-learning, papers, generative-adversarial-networks, generative-model, notation
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-04-22. Score: 2 | Accepted answer
  
  There are some definitions that may cause confusion here.
  
  In the original GANs (the first formula), the output from the discriminator connects to a sigmoid activation, the second formula is the real value from the last layer of the D model which you can find in section 3.1
  
  > note that this convention slightly differs from the standard formulation
  > in that in this case the discriminator outputs the real-valued “logits”
  
  Rewrite the first formula, consider the output from $D(x)$ is the real value, it will become:
  
  $$ \min\_{G} \max\_{D} V(D, G)=\mathbb{E}\_{\boldsymbol{x} \sim p\_{\text {data }}(\boldsymbol{x})}[\log {\frac{1}{1+exp(-D(x))}})]+\mathbb{E}\_{\boldsymbol{z} \sim p\_{\boldsymbol{z}}(\boldsymbol{z})}[\log (1-\frac{1}{1+exp(-D(G(z)))})] $$
  
  Now, it's able to proof the first and second formula is the same.
  
  
  ---
  
  