Answers.Body: <p>To calculate <span class="math-container">$p(x) = \int p(x|z) p(z) dz$</span>, you have to calculate it with <strong>all configurations</strong> of <span class="math-container">$p(x|z)$</span> and <span class="math-container">$p(z)$</span>, which scales exponentially with time. Thus, it is easier to estimate <span class="math-container">$p(x)$</span> than to calculate it correctly.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-05T04:47:35.610Z
Answers.Text: I don't think that makes sense. x is a fixed value given by data. I'm basically asking why p(x) can't be estimated through sampling the same way as the ELBO can. Also, I'm not sure what time means in your comment. I'll taking about VAEs in general.
Answers.UserDisplayName: Hanhan Li
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-05T05:01:13.160Z
Answers.Text: Time here means the time complexity you need to calculate all configurations possible with $\int p(x|z) p(z)$. When $z$ is large, it is computationally exhaustive to calculate the value. We are generally good with estimations.
Answers.UserDisplayName: Minh-Long Luu
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-05T05:47:26.047Z
Answers.Text: As I mentioned above, this shouldn't be the reason as ELBO also involves an integral over z. It's an expectation with q(z).
Answers.UserDisplayName: Hanhan Li
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-05T02:11:30.543Z
Answers.LastActivityDate: 2023-03-05T02:35:07.783Z
Answers.LastEditDate: 2023-03-05T02:35:07.783Z
Answers.UserDisplayName: Minh-Long Luu
Body: <p>Why is the ELBO of <span class="math-container">$p(x)=\int p(x|z)p(z)\mathrm{d}z$</span> easier to compute/estimate than the expression itself? Can we compute this quantity itself through sampling in the same way? I understanding that aggregating over data means taking log before summing over, but does it create any complication?</p>
  
Tags: <autoencoders><variational-autoencoder><evidence-lower-bound>
Title: Why is the variational lower bound is easier to compute than the original marginal distribution?
UserDisplayName: Hanhan Li
ThreadText: # Why is the variational lower bound is easier to compute than the original marginal distribution?
  
  (Asked by: username_0 on 2023-03-05)
  
  Why is the ELBO of $p(x)=\int p(x|z)p(z)\mathrm{d}z$ easier to compute/estimate than the expression itself? Can we compute this quantity itself through sampling in the same way? I understanding that aggregating over data means taking log before summing over, but does it create any complication?
  
  **Tags:** autoencoders, variational-autoencoder, evidence-lower-bound
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-03-05. Score: 1
  
  To calculate $p(x) = \int p(x|z) p(z) dz$, you have to calculate it with **all configurations** of $p(x|z)$ and $p(z)$, which scales exponentially with time. Thus, it is easier to estimate $p(x)$ than to calculate it correctly.
  
  > **Comments:**
  > **username_0**: I don't think that makes sense. x is a fixed value given by data. I'm basically asking why p(x) can't be estimated through sampling the same way as the ELBO can. Also, I'm not sure what time means in your comment. I'll taking about VAEs in general.  
  > **username_1**: Time here means the time complexity you need to calculate all configurations possible with $\int p(x|z) p(z)$. When $z$ is large, it is computationally exhaustive to calculate the value. We are generally good with estimations.  
  > **username_0**: As I mentioned above, this shouldn't be the reason as ELBO also involves an integral over z. It's an expectation with q(z).  
  
  ---
  
  