Answers.Body: <p>I interpret <span class="math-container">$p_{data}(y)$</span> as the empirical probability of seeing an image  <span class="math-container">$y$</span> in the training data.</p>
  <p>For example, in a typical training run, each training image is shown to the network the same number of times, so <span class="math-container">$p_{data}(y)$</span> is a discrete distribution with constant probability <span class="math-container">$p_{data}(y) = \frac 1 N$</span>. Thus, in this case:</p>
  <blockquote>
  <p>In theory, this objective can induce an output distribution over <span class="math-container">$\hat y$</span> that matches the empirical distribution <span class="math-container">$p_{data}(y)$</span>.</p>
  </blockquote>
  <p>means that training <span class="math-container">$G$</span> to minimize this objective can result in a function <span class="math-container">$G$</span> such that, if you first choose a random image <span class="math-container">$x \in X$</span>, then calculate <span class="math-container">$G(x)$</span>, the probability of obtaining any particular output image <span class="math-container">$y$</span> will also be <span class="math-container">$\frac 1 N$</span>. That is:</p>
  <p><span class="math-container">$$p(\hat y = y) = E_x[p(G(x) = y)] = \frac 1 N$$</span></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-10T15:01:27.590Z
Answers.Text: Your response is absolutely right. I would only like to remark that usually people don't actually compute $p_{data}(y) = 1/N$. Rather, they sample $y \sim p_{data}$ which corresponds to selecting a random batch from your dataset. This suffices, for instance, for computing averages of functions over y.
Answers.UserDisplayName: Eduardo Montesuma
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-08T16:03:30.550Z
Answers.LastActivityDate: 2022-06-08T16:03:30.550Z
Answers.UserDisplayName: Lee Reeves
Body: <p>In the original <a href="https://arxiv.org/pdf/1703.10593.pdf" rel="nofollow noreferrer">CycleGAN paper</a>, on the second page, there is a sentence that I didn't quite understand</p>
  <blockquote>
  <p>In theory, this objective can induce an output distribution over <span class="math-container">$\hat{y}$</span> that matches the empirical distribution <span class="math-container">$p_{\text {data }}(y)$</span> (in general, this requires <span class="math-container">$G$</span> to be stochastic) [16].</p>
  </blockquote>
  <p>What does <span class="math-container">$p_{\text {data }}(y)$</span> denote? Furthermore, I can't imagine the empirical distribution of it.</p>
  <p>In the loss functions, there is also <span class="math-container">$x \sim p_{\text {data }}(x)$</span>, but I also don't get the context there.</p>
  <p>Could anyone please elaborate further and explain this sentence to me?</p>
  
Tags: <generative-adversarial-networks><probability-distribution><notation><generator><cycle-gan>
Title: What is the meaning of $p_{\text {data }}(y)$ in the CycleGAN?
UserDisplayName: Lukas Pezzei
ThreadText: # What is the meaning of $p_{\text {data }}(y)$ in the CycleGAN?
  
  (Asked by: username_0 on 2022-06-08)
  
  In the original [CycleGAN paper](https://arxiv.org/pdf/1703.10593.pdf), on the second page, there is a sentence that I didn't quite understand
  
  > In theory, this objective can induce an output distribution over $\hat{y}$ that matches the empirical distribution $p\_{\text {data }}(y)$ (in general, this requires $G$ to be stochastic) [16].
  
  What does $p\_{\text {data }}(y)$ denote? Furthermore, I can't imagine the empirical distribution of it.
  
  In the loss functions, there is also $x \sim p\_{\text {data }}(x)$, but I also don't get the context there.
  
  Could anyone please elaborate further and explain this sentence to me?
  
  **Tags:** generative-adversarial-networks, probability-distribution, notation, generator, cycle-gan
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-06-08. Score: 1
  
  I interpret $p\_{data}(y)$ as the empirical probability of seeing an image $y$ in the training data.
  
  For example, in a typical training run, each training image is shown to the network the same number of times, so $p\_{data}(y)$ is a discrete distribution with constant probability $p\_{data}(y) = \frac 1 N$. Thus, in this case:
  
  > In theory, this objective can induce an output distribution over $\hat y$ that matches the empirical distribution $p\_{data}(y)$.
  
  means that training $G$ to minimize this objective can result in a function $G$ such that, if you first choose a random image $x \in X$, then calculate $G(x)$, the probability of obtaining any particular output image $y$ will also be $\frac 1 N$. That is:
  
  $$p(\hat y = y) = E\_x[p(G(x) = y)] = \frac 1 N$$
  
  > **Comments:**
  > **username_2**: Your response is absolutely right. I would only like to remark that usually people don't actually compute $p_{data}(y) = 1/N$. Rather, they sample $y \sim p_{data}$ which corresponds to selecting a random batch from your dataset. This suffices, for instance, for computing averages of functions over y.  
  
  ---
  
  