Answers.Body: <p>In VAE's, we want to model the distribution of images <span class="math-container">$x$</span> with some latent variable <span class="math-container">$z$</span>. Because <span class="math-container">$x$</span> is a random variable, You can think of <span class="math-container">$P(x|z)$</span> as the distribution of images <span class="math-container">$x$</span> conditioned on the random variable <span class="math-container">$z$</span>. So given a particular value of <span class="math-container">$z$</span>, we can generate a distribution over images <span class="math-container">$x$</span>.</p>
  <p>VAE's try to model images, which are themselves high dimensional 2D data. Given a 28x28 image, we already have 784 latent variables to model. We cannot visualise the distribution over all images <span class="math-container">$x$</span>. Your notation <span class="math-container">$P(x &lt; X|z)$</span> makes sense in a 1D case with a scalar value. However when considering 2D and higher, we have a problem with how we consider what is less then. if <span class="math-container">$x = (y_1,y_2)$</span> and <span class="math-container">$X = (y_3,y_4)$</span>, then is <span class="math-container">$x &lt; X$</span> if both <span class="math-container">$y_1 &lt; y_3$</span> and <span class="math-container">$y_2 &lt; y_4$</span>? (I.e all dimensions have to be less than or if only one dimension needs to be less than). When talking about high dimensional space therefore, it is not very useful to denote <span class="math-container">$P(x &lt; X|z)$</span> because of difficulty in interpreting the results.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-04-15T08:35:27.673Z
Answers.Text: I didn't think too much about notation. The point is that P(X), that is, the probability of a single point (in, maybe, a multi dimensional space), in a continuous distribution, is 0.
Answers.UserDisplayName: IttayD
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-04-14T12:54:52.830Z
Answers.LastActivityDate: 2021-04-14T12:54:52.830Z
Answers.UserDisplayName: calveeen
Answers.Body: <p>Whilst you're right that for any continuous distribution <span class="math-container">$P(X = x) = 0 \;; \forall x \in \mathcal{X}$</span> where <span class="math-container">$\mathcal{X}$</span> is there support of the distribution, they are not referring to probabilities here, rather they are referring to <a href="https://en.wikipedia.org/wiki/Probability_density_function" rel="nofollow noreferrer">density functions</a> (though this should really be denoted with a lower case <span class="math-container">$p$</span> to avoid confusion such as this).</p>
  <p><span class="math-container">$p(x|z)$</span> is a <a href="https://en.wikipedia.org/wiki/Conditional_probability_distribution" rel="nofollow noreferrer">conditional distribution</a>, which is also allowed in the continuous case -- you can also 'mix and match', i.e. <span class="math-container">$x$</span> could be continuous and <span class="math-container">$z$</span> could be discrete, and vice-versa.</p>
  <p>In the paper, all the authors are meaning when they write <span class="math-container">$p(x|z)$</span> is the density of <span class="math-container">$x$</span> conditioned on <span class="math-container">$z$</span>; in VAE's with an image application this is the conditional density of the image <span class="math-container">$x$</span> given your latent vector <span class="math-container">$z$</span>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-04-15T19:35:28.227Z
Answers.LastActivityDate: 2021-04-15T19:35:28.227Z
Answers.UserDisplayName: David
Body: <p>If <span class="math-container">$x \sim \mathcal{N}(\mu,\,\sigma^{2})$</span>, then it is a continuous variable, and therefore <span class="math-container">$P(x) = 0$</span> for any x. One can only consider things like <span class="math-container">$P(x&lt;X)$</span> to get a probability greater than 0.</p>
  <p>So what is the meaning of probabilities such as <span class="math-container">$P(x|z)$</span> in variational autoencoders? I can't think of <span class="math-container">$P(x|z)$</span> as meaning <span class="math-container">$P(x&lt;X|z)$</span>, if <span class="math-container">$x$</span> is an image, since <span class="math-container">$x&lt;X$</span> don't really make sense (all images smaller than a given one?)</p>
  
Tags: <papers><variational-autoencoder><notation>
Title: In variational autoencoders, what does p(x|z) mean?
UserDisplayName: IttayD
ThreadText: # In variational autoencoders, what does p(x|z) mean?
  
  (Asked by: username_0 on 2021-04-14)
  
  If $x \sim \mathcal{N}(\mu,\,\sigma^{2})$, then it is a continuous variable, and therefore $P(x) = 0$ for any x. One can only consider things like $P(x to get a probability greater than 0.
  
  So what is the meaning of probabilities such as $P(x|z)$ in variational autoencoders? I can't think of $P(x|z)$ as meaning $P(x, if $x$ is an image, since $x don't really make sense (all images smaller than a given one?)
  
  **Tags:** papers, variational-autoencoder, notation
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2021-04-15. Score: 3 | Accepted answer
  
  Whilst you're right that for any continuous distribution $P(X = x) = 0 \;; \forall x \in \mathcal{X}$ where $\mathcal{X}$ is there support of the distribution, they are not referring to probabilities here, rather they are referring to [density functions](https://en.wikipedia.org/wiki/Probability_density_function) (though this should really be denoted with a lower case $p$ to avoid confusion such as this).
  
  $p(x|z)$ is a [conditional distribution](https://en.wikipedia.org/wiki/Conditional_probability_distribution), which is also allowed in the continuous case -- you can also 'mix and match', i.e. $x$ could be continuous and $z$ could be discrete, and vice-versa.
  
  In the paper, all the authors are meaning when they write $p(x|z)$ is the density of $x$ conditioned on $z$; in VAE's with an image application this is the conditional density of the image $x$ given your latent vector $z$.
  
  
  ---
  
  ### Answer by username_1 on 2021-04-14. Score: 0
  
  In VAE's, we want to model the distribution of images $x$ with some latent variable $z$. Because $x$ is a random variable, You can think of $P(x|z)$ as the distribution of images $x$ conditioned on the random variable $z$. So given a particular value of $z$, we can generate a distribution over images $x$.
  
  VAE's try to model images, which are themselves high dimensional 2D data. Given a 28x28 image, we already have 784 latent variables to model. We cannot visualise the distribution over all images $x$. Your notation $P(x < X|z)$ makes sense in a 1D case with a scalar value. However when considering 2D and higher, we have a problem with how we consider what is less then. if $x = (y\_1,y\_2)$ and $X = (y\_3,y\_4)$, then is $x < X$ if both $y\_1 < y\_3$ and $y\_2 < y\_4$? (I.e all dimensions have to be less than or if only one dimension needs to be less than). When talking about high dimensional space therefore, it is not very useful to denote $P(x < X|z)$ because of difficulty in interpreting the results.
  
  > **Comments:**
  > **username_0**: I didn't think too much about notation. The point is that P(X), that is, the probability of a single point (in, maybe, a multi dimensional space), in a continuous distribution, is 0.  
  
  ---
  
  