<p>There are some definitions that may cause confusion here.</p>
<p>In the original GANs (the first formula), the output from the discriminator connects to a sigmoid activation, the second formula is the real value from the last layer of the D model which you can find in section 3.1</p>
<blockquote>
<p>note that this convention slightly differs from the standard formulation
in that in this case the discriminator outputs the real-valued “logits”</p>
</blockquote>
<p>Rewrite the first formula, consider the output from <span class="math-container">$D(x)$</span> is the real value, it will become:</p>
<p><span class="math-container">$$ \min_{G} \max_{D} V(D, G)=\mathbb{E}_{\boldsymbol{x} \sim p_{\text {data }}(\boldsymbol{x})}[\log {\frac{1}{1+exp(-D(x))}})]+\mathbb{E}_{\boldsymbol{z} \sim p_{\boldsymbol{z}}(\boldsymbol{z})}[\log (1-\frac{1}{1+exp(-D(G(z)))})] $$</span></p>
<p>Now, it's able to proof the first and second formula is the same.</p>

CC BY-SA 4.0
2022-04-22T05:37:04.027Z
2022-04-22T05:37:04.027Z
CuCaRot
<p>I am trying to understand how the loss function <a href="https://arxiv.org/pdf/1406.2661.pdf" rel="nofollow noreferrer">from the original GAN paper</a></p>
<p><span class="math-container">$$\min_{G} \max_{D} V(D, G)=\mathbb{E}_{\boldsymbol{x} \sim p_{\text {data }}(\boldsymbol{x})}[\log D(\boldsymbol{x})]+\mathbb{E}_{\boldsymbol{z} \sim p_{\boldsymbol{z}}(\boldsymbol{z})}[\log (1-D(G(\boldsymbol{z})))]$$</span></p>
<p>relates to generalization in the <a href="https://arxiv.org/pdf/1706.04156.pdf" rel="nofollow noreferrer">Nagarajan &amp; Kolter (2017) </a></p>
<p><span class="math-container">$$\min_{G} \max_{D} V(D, G)=\mathbb{E}_{\boldsymbol{x} \sim p_{\text {data }}(\boldsymbol{x})}[f(D(x))]+\mathbb{E}_{\boldsymbol{z} \sim p_{\boldsymbol{z}}(\boldsymbol{z})}[f(-D(G(z)))]$$</span></p>
<p>where supposedly the original loss function could be recovered via</p>
<p><span class="math-container">$$f(x) = -\log(1 + \exp(-x))$$</span></p>
<p>But when I try to substitute <span class="math-container">$f$</span> into the generalized version of the loss function I am not getting the original one. What would be the correct steps to recover the original loss function?</p>

<machine-learning><papers><generative-adversarial-networks><generative-model><notation>
What are the steps to derive the original GAN loss function from the generalized version?
Paul
# What are the steps to derive the original GAN loss function from the generalized version?

(Asked by: username_0 on 2022-04-20)

I am trying to understand how the loss function [from the original GAN paper](https://arxiv.org/pdf/1406.2661.pdf)

$$\min\_{G} \max\_{D} V(D, G)=\mathbb{E}\_{\boldsymbol{x} \sim p\_{\text {data }}(\boldsymbol{x})}[\log D(\boldsymbol{x})]+\mathbb{E}\_{\boldsymbol{z} \sim p\_{\boldsymbol{z}}(\boldsymbol{z})}[\log (1-D(G(\boldsymbol{z})))]$$

relates to generalization in the [Nagarajan & Kolter (2017)](https://arxiv.org/pdf/1706.04156.pdf)

$$\min\_{G} \max\_{D} V(D, G)=\mathbb{E}\_{\boldsymbol{x} \sim p\_{\text {data }}(\boldsymbol{x})}[f(D(x))]+\mathbb{E}\_{\boldsymbol{z} \sim p\_{\boldsymbol{z}}(\boldsymbol{z})}[f(-D(G(z)))]$$

where supposedly the original loss function could be recovered via

$$f(x) = -\log(1 + \exp(-x))$$

But when I try to substitute $f$ into the generalized version of the loss function I am not getting the original one. What would be the correct steps to recover the original loss function?

**Tags:** machine-learning, papers, generative-adversarial-networks, generative-model, notation


---

## 1 answers

### Answer by username_1 on 2022-04-22. Score: 2 | Accepted answer

There are some definitions that may cause confusion here.

In the original GANs (the first formula), the output from the discriminator connects to a sigmoid activation, the second formula is the real value from the last layer of the D model which you can find in section 3.1

> note that this convention slightly differs from the standard formulation
> in that in this case the discriminator outputs the real-valued “logits”

Rewrite the first formula, consider the output from $D(x)$ is the real value, it will become:

$$ \min\_{G} \max\_{D} V(D, G)=\mathbb{E}\_{\boldsymbol{x} \sim p\_{\text {data }}(\boldsymbol{x})}[\log {\frac{1}{1+exp(-D(x))}})]+\mathbb{E}\_{\boldsymbol{z} \sim p\_{\boldsymbol{z}}(\boldsymbol{z})}[\log (1-\frac{1}{1+exp(-D(G(z)))})] $$

Now, it's able to proof the first and second formula is the same.


---

