Answers.Body: <p>When <span class="math-container">$h$</span> is conditioned on <span class="math-container">$x$</span> there is a function <span class="math-container">$Q_\theta$</span> that maps <span class="math-container">$x$</span> onto a distribution for h. However, when not conditioned on x there exists just a prior p(h), which is independent of <span class="math-container">$\theta$</span> and <span class="math-container">$\phi$</span>.</p>
  <p>Easy to see when you factorise the joint <span class="math-container">$P_\theta(x|h)P(h)$</span></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-16T19:38:54.987Z
Answers.Text: Thanks, but I don't think this answers my question per se. I am not entirely sure why the mapping $Q_{\theta}$ allows the VAE to approximate arbitrary distributions using the Gaussian latent space.
Answers.UserDisplayName: Decaying Tails
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-17T15:20:04.377Z
Answers.Text: The likelihood $P(x)$ is typically complex and has no closed form, but when conditioned on latent variable h it usually has a simpler form written as $P_\theta(x|h)$. Similarly, we don't know the distribution P(h|x) and instead  approximate it as $Q(h|x)$. 
  Updating $\phi$ causes Q(h|x) to tend to P(h|x) based on standard VAE theory but this is limited to how flexible the parameters of the mapping Q is.
Answers.UserDisplayName: quest ions
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-17T15:24:36.340Z
Answers.Text: Also, if this is your main question could make this a bit clearer in the post
Answers.UserDisplayName: quest ions
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-17T16:48:59.367Z
Answers.Text: Sorry, this is my bad, I got confused with a different question that I posted on the site; ignore my first comment please.
  
  That said, I still don't understand your response. My issue is that when we backpropagate through the VAE, we have that $\nabla_{\phi} P_{\theta}(x|h)$ is non-zero since the gradient flows through the network; however, when moving the gradients within the expectation (like the summation from Appendix B shown in my question), we treat $P_{\theta}(x|h)$ as a constant w.r.t $\nabla_{\phi}$, which seems to contradict our backpropagation step.
Answers.UserDisplayName: Decaying Tails
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-17T17:25:25.857Z
Answers.Text: h in $P_\theta(x|h)$ comes from a prior distribution, p(h), which is independent of $\phi$. They use the form $p_\theta(x,h)$ which can make things a bit confusing but if they factored it into the form p(h|x)p(x) we just get p(x) back which would be pointless. Also note that $p(h|x) \ne q(h|x)$ since q is an approximation. It might also help understanding that Q isn't a true mapping of h to x it is an approximation
Answers.UserDisplayName: quest ions
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-17T23:37:02.213Z
Answers.Text: I see, so, for example, if we are finding
  $\nabla_{\phi} \int_{\Omega_{h}} p_{\theta}(x|h) q_{\phi}(h|x) \operatorname{dh}$, can we treat $p_{\theta}(x|h)$ as a constant w.r.t $\phi$, since we are integrating over the latent space $\Omega_{h}$? 
  
  I don't see how this implies that we are sampling from the prior distribution, since, in the actual network, we sample $h$ using the parameters that the decoder $q_{\phi}(h|x)$ outputs, so I don't see how we can just assume that $p_{\theta}(x|h)$ is constant w.r.t  to $\phi$, since ultimately it is a function of $h \sim q_{\phi}(h|x)$.
Answers.UserDisplayName: Decaying Tails
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-21T14:34:20.207Z
Answers.Text: Ok so when doing gradient descent, samples $z$, are taken from the distribution $q_\phi(h|x)$. This makes $z$ a constant that can't be propagated through implicitly. The other term is a function expressed explicitly in terms of $\phi$ hence the derivative be taken here. Also note that despite the fact the decoder section of the VAE is trained on q(h|x) in practice it can be used with noise ($p(h)$) as inputs for image synthesis
Answers.UserDisplayName: quest ions
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-25T07:58:38.067Z
Answers.Text: @Joel does this answer your question?
Answers.UserDisplayName: quest ions
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-16T09:55:39.810Z
Answers.LastActivityDate: 2023-08-16T09:55:39.810Z
Answers.UserDisplayName: quest ions
Body: <p>I am confused as to when to hold certain parameters constant in a VAE. I will explain with a concrete example.</p>
  <p>We can write <span class="math-container">$\operatorname{ELBO}(\phi, \theta) = \mathbb{E}_{q_{\phi}(z)}\left[\log 
  \left(p_{\theta}(x| z)\right)\right] - D_{\operatorname{KL}}[q_{\phi}(z) | p(z)]$</span>, where we wish to find <span class="math-container">$\nabla_{\phi, \theta}\operatorname{ELBO}(\phi, \theta)$</span>. We can take the gradient of the KL divergence quite easily since it can be analytically solved.</p>
  <p>My issue is with the graident <span class="math-container">$\nabla_{\phi, \theta}\mathbb{E}_{q_{\phi}(z)}\left[\log 
  \left(p_{\theta}(x| z)\right)\right]$</span>. I am assuming that the expectation is intractable and therefore we can use a Monte Carlo (MC) approximation and instead find
  <span class="math-container">$$
  \nabla_{\phi, \theta}\left(\frac{1}{L}\sum_{l=1}^{L}\log p_{\theta}\left(x|z^{(i)}\right)\right)
  $$</span>
  where we use <span class="math-container">$L$</span> samples for the MC approximation. From my understanding, the gradient of this term w.r.t <span class="math-container">$\phi$</span> should be non-zero since changing <span class="math-container">$\phi$</span> should change <span class="math-container">$z$</span>, which would change the above term. However, when I look at a derivations for gradient estimators such as the <em>Score Function Estimator</em>, I see that <span class="math-container">$\theta$</span> is treated as a constant w.r.t <span class="math-container">$\phi$</span>. Example from Appendix B of the <a href="https://www.cs.toronto.edu/%7Eamnih/papers/nvil.pdf" rel="nofollow noreferrer">linked paper</a>:</p>
  <p><span class="math-container">$$
  \nabla_{\phi}\sum_{h}Q_{\phi}(h|x)\log P_{\theta}(x, h) \implies \sum_{h}\log P_{\theta}(x, h)\nabla_{\phi}Q_{\phi}(h|x)
  $$</span>
  I am not sure how to connect the two differences here. One potential explanation is that the latent <span class="math-container">$h$</span> is being treated fixed in the above equation; however, I don't see why this should be the case since <span class="math-container">$h$</span> is a function of the parameters <span class="math-container">$\phi$</span> and so changing <span class="math-container">$\phi$</span> would in turn change <span class="math-container">$h$</span> and thus the value of <span class="math-container">$\log P_{\theta}(x, h)$</span>?</p>
  
Tags: <neural-networks><machine-learning><backpropagation><variational-autoencoder><evidence-lower-bound>
Title: Confusion over taking gradients in Variational Autoencoders (VAE)
UserDisplayName: Decaying Tails
ThreadText: # Confusion over taking gradients in Variational Autoencoders (VAE)
  
  (Asked by: username_0 on 2023-08-11)
  
  I am confused as to when to hold certain parameters constant in a VAE. I will explain with a concrete example.
  
  We can write $\operatorname{ELBO}(\phi, \theta) = \mathbb{E}\_{q\_{\phi}(z)}\left[\log
  \left(p\_{\theta}(x| z)\right)\right] - D\_{\operatorname{KL}}[q\_{\phi}(z) | p(z)]$, where we wish to find $\nabla\_{\phi, \theta}\operatorname{ELBO}(\phi, \theta)$. We can take the gradient of the KL divergence quite easily since it can be analytically solved.
  
  My issue is with the graident $\nabla\_{\phi, \theta}\mathbb{E}\_{q\_{\phi}(z)}\left[\log
  \left(p\_{\theta}(x| z)\right)\right]$. I am assuming that the expectation is intractable and therefore we can use a Monte Carlo (MC) approximation and instead find
  $$
  \nabla\_{\phi, \theta}\left(\frac{1}{L}\sum\_{l=1}^{L}\log p\_{\theta}\left(x|z^{(i)}\right)\right)
  $$
  where we use $L$ samples for the MC approximation. From my understanding, the gradient of this term w.r.t $\phi$ should be non-zero since changing $\phi$ should change $z$, which would change the above term. However, when I look at a derivations for gradient estimators such as the *Score Function Estimator*, I see that $\theta$ is treated as a constant w.r.t $\phi$. Example from Appendix B of the [linked paper](https://www.cs.toronto.edu/%7Eamnih/papers/nvil.pdf):
  
  $$
  \nabla\_{\phi}\sum\_{h}Q\_{\phi}(h|x)\log P\_{\theta}(x, h) \implies \sum\_{h}\log P\_{\theta}(x, h)\nabla\_{\phi}Q\_{\phi}(h|x)
  $$
  I am not sure how to connect the two differences here. One potential explanation is that the latent $h$ is being treated fixed in the above equation; however, I don't see why this should be the case since $h$ is a function of the parameters $\phi$ and so changing $\phi$ would in turn change $h$ and thus the value of $\log P\_{\theta}(x, h)$?
  
  **Tags:** neural-networks, machine-learning, backpropagation, variational-autoencoder, evidence-lower-bound
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-08-16. Score: 0
  
  When $h$ is conditioned on $x$ there is a function $Q\_\theta$ that maps $x$ onto a distribution for h. However, when not conditioned on x there exists just a prior p(h), which is independent of $\theta$ and $\phi$.
  
  Easy to see when you factorise the joint $P\_\theta(x|h)P(h)$
  
  > **Comments:**
  > **username_0**: Thanks, but I don't think this answers my question per se. I am not entirely sure why the mapping $Q_{\theta}$ allows the VAE to approximate arbitrary distributions using the Gaussian latent space.  
  > **username_1**: The likelihood $P(x)$ is typically complex and has no closed form, but when conditioned on latent variable h it usually has a simpler form written as $P_\theta(x|h)$. Similarly, we don't know the distribution P(h|x) and instead  approximate it as $Q(h|x)$.  Updating $\phi$ causes Q(h|x) to tend to P(h|x) based on standard VAE theory but this is limited to how flexible the parameters of the mapping Q is.  
  > **username_1**: Also, if this is your main question could make this a bit clearer in the post  
  > **username_0**: Sorry, this is my bad, I got confused with a different question that I posted on the site; ignore my first comment please.  That said, I still don't understand your response. My issue is that when we backpropagate through the VAE, we have that $\nabla_{\phi} P_{\theta}(x|h)$ is non-zero since the gradient flows through the network; however, when moving the gradients within the expectation (like the summation from Appendix B shown in my question), we treat $P_{\theta}(x|h)$ as a constant w.r.t $\nabla_{\phi}$, which seems to contradict our backpropagation step.  
  > **username_1**: h in $P_\theta(x|h)$ comes from a prior distribution, p(h), which is independent of $\phi$. They use the form $p_\theta(x,h)$ which can make things a bit confusing but if they factored it into the form p(h|x)p(x) we just get p(x) back which would be pointless. Also note that $p(h|x) \ne q(h|x)$ since q is an approximation. It might also help understanding that Q isn't a true mapping of h to x it is an approximation  
  > **username_0**: I see, so, for example, if we are finding $\nabla_{\phi} \int_{\Omega_{h}} p_{\theta}(x|h) q_{\phi}(h|x) \operatorname{dh}$, can we treat $p_{\theta}(x|h)$ as a constant w.r.t $\phi$, since we are integrating over the latent space $\Omega_{h}$?   I don't see how this implies that we are sampling from the prior distribution, since, in the actual network, we sample $h$ using the parameters that the decoder $q_{\phi}(h|x)$ outputs, so I don't see how we can just assume that $p_{\theta}(x|h)$ is constant w.r.t  to $\phi$, since ultimately it is a function of $h \sim q_{\phi}(h|x)$.  
  > **username_1**: Ok so when doing gradient descent, samples $z$, are taken from the distribution $q_\phi(h|x)$. This makes $z$ a constant that can't be propagated through implicitly. The other term is a function expressed explicitly in terms of $\phi$ hence the derivative be taken here. Also note that despite the fact the decoder section of the VAE is trained on q(h|x) in practice it can be used with noise ($p(h)$) as inputs for image synthesis  
  > **username_1**: @Joel does this answer your question?  
  
  ---
  
  