Answers.Body: <p>I don't see where it's implied that G is a probability distribution. G is a function, whose output conditioned on one variable <em>has</em> a probability distribution, but it isn't one.</p>
  <p>z is random noise which is G's source of randomness. y is something that isn't random. We call G over and over with the same y and different random z's and look at the distribution of the output.</p>
  <p>For example, here are some outputs of one possible G(z,y), for different z, when y=3. z is not displayed, only G(z,y). This G outputs one digit.</p>
  <p>2262662626626262626262626626262262622226626622626662626666262626262626262</p>
  <p>These digits have a probability distribution. About half of them are 2 and about half of them are 6. P(G(z)=2|y=3) = 0.479 and P(G(z)=6|y=3) = 0.521 (approximately). Even though neither 2 nor 6 is a valid probability.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-08-05T12:13:17.820Z
Answers.Text: So, the notation has to be $G(z,y)$, Then what might be the reason for using $G(z|y)$?
Answers.UserDisplayName: hanugm
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-08-05T13:27:26.630Z
Answers.Text: @hanugm Notation does not matter. Notation does not have to be a certain way. It seems that both z and y are inputs to G. They probably write G(z|y) because it reminds you of conditional probability.
Answers.UserDisplayName: Criticize SE actions means ban
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-08-05T12:07:04.073Z
Answers.LastActivityDate: 2021-08-05T12:07:04.073Z
Answers.UserDisplayName: Criticize SE actions means ban
Body: <p>In probability, we have two types of probability functions: unconditional probability <span class="math-container">$p(x)$</span> and conditional probability <span class="math-container">$p(x | y)$</span>. Both are fundamentally different and the latter can be obtained by the following equation</p>
  <p><span class="math-container">$$p(x|y) = \dfrac{p(x, y)}{p(y)} \text{          provided   } p(y) \ne 0$$</span></p>
  <p>I never heard of formal definition for conditioning except for conditional probability function.</p>
  <p>But in case of neural networks, I came across the <a href="https://arxiv.org/pdf/1411.1784.pdf" rel="nofollow noreferrer">notion of conditioning</a>.</p>
  <p><span class="math-container">$$\min_G \max_DV(D, G) = \mathbb{E}_{x ∼ P_{data}}[\log D(x|y)] +  \mathbb{E}_{z ∼ p_z}[log (1 - D(G(z|y)))]$$</span></p>
  <p>Since neural network <span class="math-container">$D$</span> is intended to implement a probability function, we can at-least think about conditioning on an input. But the neural network <span class="math-container">$G$</span> is not intended to implement probability function. <span class="math-container">$G$</span> is intended to provide datasamples by learning an underlying probability distribution whose output is not in the range <span class="math-container">$[0, 1]$</span>.</p>
  <p>Does <span class="math-container">$G$</span> obey the laws of probability? If yes, how, since its output is not restricted to <span class="math-container">$[0, 1]$</span>?  If no, then why the authors use the notation of conditional probability for <span class="math-container">$G$</span> also?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-08-05T10:36:16.130Z
Comments.Text: Why would the output have to be in the range [0,1]? Who said G's output is a probability?
Comments.UserDisplayName: Criticize SE actions means ban
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-08-05T10:36:50.753Z
Comments.Text: G is more like a function for sampling from a probability distribution
Comments.UserDisplayName: Criticize SE actions means ban
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-08-05T10:46:38.250Z
Comments.Text: @user253751 Confusion is due to the notation of conditional probability $G(x|y)$.
Comments.UserDisplayName: hanugm
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-08-05T10:53:24.983Z
Comments.Text: @user253751 I didn't see any function till now of the form f(x|y) except conditional probability distribution function p(x|y).
Comments.UserDisplayName: hanugm
Tags: <generative-adversarial-networks><probability><notation>
Title: Does generator in conditonal GAN obey probability laws?
UserDisplayName: hanugm
ThreadText: # Does generator in conditonal GAN obey probability laws?
  
  (Asked by: username_0 on 2021-08-04)
  
  In probability, we have two types of probability functions: unconditional probability $p(x)$ and conditional probability $p(x | y)$. Both are fundamentally different and the latter can be obtained by the following equation
  
  $$p(x|y) = \dfrac{p(x, y)}{p(y)} \text{ provided } p(y) \ne 0$$
  
  I never heard of formal definition for conditioning except for conditional probability function.
  
  But in case of neural networks, I came across the [notion of conditioning](https://arxiv.org/pdf/1411.1784.pdf).
  
  $$\min\_G \max\_DV(D, G) = \mathbb{E}\_{x ∼ P\_{data}}[\log D(x|y)] + \mathbb{E}\_{z ∼ p\_z}[log (1 - D(G(z|y)))]$$
  
  Since neural network $D$ is intended to implement a probability function, we can at-least think about conditioning on an input. But the neural network $G$ is not intended to implement probability function. $G$ is intended to provide datasamples by learning an underlying probability distribution whose output is not in the range $[0, 1]$.
  
  Does $G$ obey the laws of probability? If yes, how, since its output is not restricted to $[0, 1]$? If no, then why the authors use the notation of conditional probability for $G$ also?
  
  **Tags:** generative-adversarial-networks, probability, notation
  
  > **Comments:**
  > **username_1**: Why would the output have to be in the range [0,1]? Who said G's output is a probability?  
  > **username_1**: G is more like a function for sampling from a probability distribution  
  > **username_0**: @user253751 Confusion is due to the notation of conditional probability $G(x|y)$.  
  > **username_0**: @user253751 I didn't see any function till now of the form f(x|y) except conditional probability distribution function p(x|y).  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-08-05. Score: 1
  
  I don't see where it's implied that G is a probability distribution. G is a function, whose output conditioned on one variable *has* a probability distribution, but it isn't one.
  
  z is random noise which is G's source of randomness. y is something that isn't random. We call G over and over with the same y and different random z's and look at the distribution of the output.
  
  For example, here are some outputs of one possible G(z,y), for different z, when y=3. z is not displayed, only G(z,y). This G outputs one digit.
  
  2262662626626262626262626626262262622226626622626662626666262626262626262
  
  These digits have a probability distribution. About half of them are 2 and about half of them are 6. P(G(z)=2|y=3) = 0.479 and P(G(z)=6|y=3) = 0.521 (approximately). Even though neither 2 nor 6 is a valid probability.
  
  > **Comments:**
  > **username_0**: So, the notation has to be $G(z,y)$, Then what might be the reason for using $G(z|y)$?  
  > **username_1**: @username_0 Notation does not matter. Notation does not have to be a certain way. It seems that both z and y are inputs to G. They probably write G(z|y) because it reminds you of conditional probability.  
  
  ---
  
  