Answers.Body: <p>Your mistake is that you think that the referenced <span class="math-container">$V(D,G)$</span> is the definition of the cross-entropy! Indeed, the cross-entropy is defined based on the negative value of the <span class="math-container">$V(D,G)$</span>. Hence, if you consider the minus behind the <span class="math-container">$V(D,G)$</span> (<span class="math-container">$-V(D,G)$</span>) the sentence will be meaningful.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-24T13:22:07.990Z
Answers.LastActivityDate: 2021-12-10T16:15:00.577Z
Answers.LastEditDate: 2021-12-10T16:15:00.577Z
Answers.UserDisplayName: OmG
Body: <p>In his original <a href="https://arxiv.org/pdf/1406.2661v1.pdf" rel="nofollow noreferrer">GAN paper</a> Goodfellow gives a game theoretic perspective for GANs:</p>
  
  <p><span class="math-container">\begin{equation} 
  \underset{G}{\min}\, \underset{D}{\max}\, V\left(D,G \right) = 
  \mathbb{E}_{x\sim\mathit{p}_{\textrm{data}}\left(x \right)} \left[\textrm{log}\, D \left(x \right) \right]
  + \mathbb{E}_{z\sim\mathit{p}_{\textrm{z}}\left(z \right)} \left[\textrm{log} \left(1 - D \left(G \left(z \right)\right)\right) \right]
  \end{equation}</span></p>
  
  <p>I think I understand this formula, at least it makes sense to me.
  What I don't understand is that he writes in his <a href="https://arxiv.org/pdf/1701.00160.pdf" rel="nofollow noreferrer">NIPS tutorial</a>:</p>
  
  <blockquote>
    <p>In the minimax game, the discriminator minimizes a cross-entropy, but the generator maximizes the same cross-entropy.</p>
  </blockquote>
  
  <p>Why does he write that the discriminator <strong>minimizes</strong> the cross-entropy while the generator <strong>maximizes</strong> it? Shouldn't it be the other way around? At least that is how I understand <span class="math-container">$\underset{G}{\min}\, \underset{D}{\max}\, V\left(D,G \right)$</span>.</p>
  
  <p>I guess this shows that I have a fundamental error in my understanding. Could anyone clarify what I'm missing here?</p>
  
Tags: <neural-networks><machine-learning><math><generative-adversarial-networks><generative-model>
Title: Why does the discriminator minimize the cross-entropy while the generator maximize it?
UserDisplayName: bw0248
ThreadText: # Why does the discriminator minimize the cross-entropy while the generator maximize it?
  
  (Asked by: username_0 on 2019-10-24)
  
  In his original [GAN paper](https://arxiv.org/pdf/1406.2661v1.pdf) Goodfellow gives a game theoretic perspective for GANs:
  
  \begin{equation}
  \underset{G}{\min}\, \underset{D}{\max}\, V\left(D,G \right) =
  \mathbb{E}\_{x\sim\mathit{p}\_{\textrm{data}}\left(x \right)} \left[\textrm{log}\, D \left(x \right) \right]
  + \mathbb{E}\_{z\sim\mathit{p}\_{\textrm{z}}\left(z \right)} \left[\textrm{log} \left(1 - D \left(G \left(z \right)\right)\right) \right]
  \end{equation}
  
  I think I understand this formula, at least it makes sense to me.
  What I don't understand is that he writes in his [NIPS tutorial](https://arxiv.org/pdf/1701.00160.pdf):
  
  > In the minimax game, the discriminator minimizes a cross-entropy, but the generator maximizes the same cross-entropy.
  
  Why does he write that the discriminator **minimizes** the cross-entropy while the generator **maximizes** it? Shouldn't it be the other way around? At least that is how I understand $\underset{G}{\min}\, \underset{D}{\max}\, V\left(D,G \right)$.
  
  I guess this shows that I have a fundamental error in my understanding. Could anyone clarify what I'm missing here?
  
  **Tags:** neural-networks, machine-learning, math, generative-adversarial-networks, generative-model
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-10-24. Score: 1 | Accepted answer
  
  Your mistake is that you think that the referenced $V(D,G)$ is the definition of the cross-entropy! Indeed, the cross-entropy is defined based on the negative value of the $V(D,G)$. Hence, if you consider the minus behind the $V(D,G)$ ($-V(D,G)$) the sentence will be meaningful.
  
  
  ---
  
  