Answers.Body: <p>&quot;<em>What is: x, y, P(x), P(y), P(x|y), P(y|x) in this example?</em>&quot; I don't really see where you are referring to them in the question, so I'll answer with the general interpretation in ML:</p>
  <ul>
  <li><span class="math-container">$P(x)$</span>: also known as data distribution, it's the density that describes some data... take your example, if your bins are described by height and width, you can assume that maybe the height is described by <span class="math-container">$H\sim U(1,10)$</span> (uniform) and the width by <span class="math-container">$W \sim U(1,5)$</span>, and supposing that they are independent, then the joint distribution of <span class="math-container">$H$</span> and <span class="math-container">$W$</span> is <span class="math-container">$P(x)$</span> (now, if you are dealing with images, you have to consider the joint distribution of all pixels, so it's a very high dimensional distribution)</li>
  <li><span class="math-container">$P(y)$</span> also known as target distribution. This can also be considered as &quot;prior&quot;, since in discriminative task, you try to model <span class="math-container">$P(y|x)$</span>... it describes your belief of the target distribution given no additional information... take for example a task where you try to regress the weight of a person, then you know that generally speaking, weight is distributed as a Gaussian (look up statistics online about this and you will see that Bell curve) centered at some weight</li>
  <li><span class="math-container">$P(y|x)$</span>, your conditional probability that you want to learn with discriminative tasks. It means _&quot;Given these informations <span class="math-container">$x$</span>, how likely it <span class="math-container">$y$</span> to be some value&quot;... take the weight regression task, say generally that the weight is distributed as <span class="math-container">$N(50,10)$</span>, so given no additional information, the best guess is to say that a person will weight 50kg... however, if I additionally tell you that that persone is 190cm tall, then you know that that person will weight more than the average most likely, so your belief on his weight might shift to <span class="math-container">$N(70,5)$</span>... this &quot;shift&quot; is the information brought by <span class="math-container">$x$</span>, thus <span class="math-container">$p(y|x)$</span> is just a classification/regression etc etc</li>
  <li><span class="math-container">$P(x|y)$</span> is the other way around.. if we consider images, <span class="math-container">$Y$</span> might be for example a the digit you want to see in the image, and <span class="math-container">$x$</span> the image you want to generate, then <span class="math-container">$p(y|x)$</span> is just &quot;which images are more likely to be correct given that I want to see the digit <span class="math-container">$y$</span> printed on it&quot;... you can consider that <span class="math-container">$p(x)$</span> is already a distribution and <span class="math-container">$p(x|y)$</span> just takes the interesting part of it where there is <span class="math-container">$y$</span> printed on the image</li>
  </ul>
  <p>so, in your example, P(x) is just the distribution of the images, P(y) is the distribution of some classification with those bins (say if a cookie will fit in it, it's your general belief wether a cookie fits in a random bin or not), P(y|x) is the &quot;given this image of a bin will a cookie fit in that&quot;, and P(x|y) is &quot;generate a picture of a possible bin such that this y cookie fits in&quot;</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-20T10:36:37.047Z
Answers.Text: Edited/rectified the question to better reflect the intent. Thanks for taking time to explain. Could you please update the answer accordingly. I wish to understand what the probability terms represent in the tin bucket example.
Answers.UserDisplayName: Prakhar
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-20T12:11:46.470Z
Answers.Text: @Prakhar I still don't quite get what you are referring to with those letters, however I have a sense that you are probably referring to Variational Autoencoder, where y would be z/h depending on the book you are reading?
Answers.UserDisplayName: Alberto
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-20T09:53:28.537Z
Answers.LastActivityDate: 2023-08-20T09:53:28.537Z
Answers.UserDisplayName: Alberto
Body: <h3>Background</h3>
  <p><strong>Generative modeling</strong> <br>
  Generative modeling aims to model the probability of observing an observation <code>x</code>.
  <span class="math-container">$$
  p(x) = \frac{p(y\cap x)}{p(y|x)}
  $$</span></p>
  <p><strong>Representation Learning</strong></p>
  <p>Instead of trying to model the <strong>high-dimensional sample space</strong> directly, we describe each observation in the training set using some <strong>lower-dimensional latent space</strong> and then learn a mapping function that can take a point in the latent space and map it to a point in the original domain</p>
  <hr />
  <h3>Context:</h3>
  <p>Let’s suppose we have a training set consisting of grayscale images of biscuit tins
  <a href="https://i.sstatic.net/QFgfn.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/QFgfn.png" alt="enter image description here" /></a></p>
  <p>we can convert each image of a tin to a point in a latent space of just two dimensions. We would first need to establish two latent space dimensions that best describe this dataset(height, width), then learn the mapping function <strong>f</strong> that can take a point in this space and map it to a grayscale biscuit tin image.
  <a href="https://i.sstatic.net/yIQMc.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/yIQMc.png" alt="enter image description here" /></a></p>
  <hr />
  <h3>Question:</h3>
  <p>What is:</p>
  <p><code>P(y|x)</code>: is it possibility of point lying in Latent space given it is a point in (high dimensional) pixel space? <br></p>
  <p>Or for that matter, what do <code>x, y, P(x), P(y), P(x|y)</code> really mean/represent here?</p>
  <p><em>P.S.: this is referred from the book Generative Deep Learning 2<sup>nd</sup> edition By David Foster, O'Reilly Publication.</em></p>
  
Tags: <generative-model><probability-theory><conditional-probability>
Title: What is x, y, p(x), p(y) in generative model domain?
UserDisplayName: Prakhar
ThreadText: # What is x, y, p(x), p(y) in generative model domain?
  
  (Asked by: username_0 on 2023-08-20)
  
  ### Background
  
  **Generative modeling**   
  Generative modeling aims to model the probability of observing an observation `x`.
  $$
  p(x) = \frac{p(y\cap x)}{p(y|x)}
  $$
  
  **Representation Learning**
  
  Instead of trying to model the **high-dimensional sample space** directly, we describe each observation in the training set using some **lower-dimensional latent space** and then learn a mapping function that can take a point in the latent space and map it to a point in the original domain
  
  ---
  
  ### Context:
  
  Let’s suppose we have a training set consisting of grayscale images of biscuit tins
  [![enter image description here](https://i.sstatic.net/QFgfn.png)](https://i.sstatic.net/QFgfn.png)
  
  we can convert each image of a tin to a point in a latent space of just two dimensions. We would first need to establish two latent space dimensions that best describe this dataset(height, width), then learn the mapping function **f** that can take a point in this space and map it to a grayscale biscuit tin image.
  [![enter image description here](https://i.sstatic.net/yIQMc.png)](https://i.sstatic.net/yIQMc.png)
  
  ---
  
  ### Question:
  
  What is:
  
  `P(y|x)`: is it possibility of point lying in Latent space given it is a point in (high dimensional) pixel space?
  
  Or for that matter, what do `x, y, P(x), P(y), P(x|y)` really mean/represent here?
  
  *P.S.: this is referred from the book Generative Deep Learning 2nd edition By David Foster, O'Reilly Publication.*
  
  **Tags:** generative-model, probability-theory, conditional-probability
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-08-20. Score: 1
  
  "*What is: x, y, P(x), P(y), P(x|y), P(y|x) in this example?*" I don't really see where you are referring to them in the question, so I'll answer with the general interpretation in ML:
  
  * $P(x)$: also known as data distribution, it's the density that describes some data... take your example, if your bins are described by height and width, you can assume that maybe the height is described by $H\sim U(1,10)$ (uniform) and the width by $W \sim U(1,5)$, and supposing that they are independent, then the joint distribution of $H$ and $W$ is $P(x)$ (now, if you are dealing with images, you have to consider the joint distribution of all pixels, so it's a very high dimensional distribution)
  * $P(y)$ also known as target distribution. This can also be considered as "prior", since in discriminative task, you try to model $P(y|x)$... it describes your belief of the target distribution given no additional information... take for example a task where you try to regress the weight of a person, then you know that generally speaking, weight is distributed as a Gaussian (look up statistics online about this and you will see that Bell curve) centered at some weight
  * $P(y|x)$, your conditional probability that you want to learn with discriminative tasks. It means \_"Given these informations $x$, how likely it $y$ to be some value"... take the weight regression task, say generally that the weight is distributed as $N(50,10)$, so given no additional information, the best guess is to say that a person will weight 50kg... however, if I additionally tell you that that persone is 190cm tall, then you know that that person will weight more than the average most likely, so your belief on his weight might shift to $N(70,5)$... this "shift" is the information brought by $x$, thus $p(y|x)$ is just a classification/regression etc etc
  * $P(x|y)$ is the other way around.. if we consider images, $Y$ might be for example a the digit you want to see in the image, and $x$ the image you want to generate, then $p(y|x)$ is just "which images are more likely to be correct given that I want to see the digit $y$ printed on it"... you can consider that $p(x)$ is already a distribution and $p(x|y)$ just takes the interesting part of it where there is $y$ printed on the image
  
  so, in your example, P(x) is just the distribution of the images, P(y) is the distribution of some classification with those bins (say if a cookie will fit in it, it's your general belief wether a cookie fits in a random bin or not), P(y|x) is the "given this image of a bin will a cookie fit in that", and P(x|y) is "generate a picture of a possible bin such that this y cookie fits in"
  
  > **Comments:**
  > **username_0**: Edited/rectified the question to better reflect the intent. Thanks for taking time to explain. Could you please update the answer accordingly. I wish to understand what the probability terms represent in the tin bucket example.  
  > **username_1**: @Prakhar I still don't quite get what you are referring to with those letters, however I have a sense that you are probably referring to Variational Autoencoder, where y would be z/h depending on the book you are reading?  
  
  ---
  
  