Answers.Body: <p>Indeed this sounds very confusing until you expand the product <span class="math-container">$\Pi$</span> terms as outlined below:
  <span class="math-container">$$\frac{p_{\theta}(x_1|x_2)q(x_1|x_0)p_{\theta}(x_2|x_3)q(x_2|x_0)...p_{\theta}(x_{T-1}|x_T)q(x_{T-1}|x_0)}{q(x_1|x_2,x_0)q(x_2|x_0)...q(x_{T-2}|x_{T-1},x_0)q(x_{T-1}|x_0)q(x_{T-1}|x_T,x_0)q(x_T|x_0)}$$</span></p>
  <p>Therefore now it's obvious that the cancelation is from all alternating <span class="math-container">$q$</span> terms with only one conditioned variable <span class="math-container">$x_0$</span>  in the numerator and denominator, respectively, <em>except</em> the first and last terms <span class="math-container">$\frac{q(x_1|x_0)}{q(x_T|x_0)}$</span> still remain. That's why in your reference the next step in show me proof section there appears one additional middle term <span class="math-container">$+\log \frac{q(x_1|x_0)}{q(x_T|x_0)}$</span>. The author perhaps should expand like here to help reader's clear comprehension considering he's already written so many detailed steps.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-11-25T05:30:33.217Z
Answers.LastActivityDate: 2024-11-25T05:30:33.217Z
Answers.UserDisplayName: cinch
Body: <p>I was looking at the derivation of the ELBO for diffusion models from this <a href="https://calvinyluo.com/2022/08/26/diffusion-tutorial.html#mjx-eqn%3Aeq%3A19" rel="nofollow noreferrer">website</a> and I'm a little bit confused as to why they cancelled these two terms out.</p>
  <p><a href="https://i.sstatic.net/F0kxJ4dV.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/F0kxJ4dV.png" alt="enter image description here" /></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-11-24T04:24:10.640Z
Comments.Text: Where you see this derivation in your linked website? which equation number?
Comments.UserDisplayName: cinch
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-11-24T15:30:38.043Z
Comments.Text: 37 in the show me the proof part
Comments.UserDisplayName: henrysilver
Tags: <machine-learning><math><diffusion-models><evidence-lower-bound>
Title: Why are these 2 terms canceled out in the ELBO derivation for diffusion models?
UserDisplayName: henrysilver
ThreadText: # Why are these 2 terms canceled out in the ELBO derivation for diffusion models?
  
  (Asked by: username_0 on 2024-11-24)
  
  I was looking at the derivation of the ELBO for diffusion models from this [website](https://calvinyluo.com/2022/08/26/diffusion-tutorial.html#mjx-eqn%3Aeq%3A19) and I'm a little bit confused as to why they cancelled these two terms out.
  
  [![enter image description here](https://i.sstatic.net/F0kxJ4dV.png)](https://i.sstatic.net/F0kxJ4dV.png)
  
  **Tags:** machine-learning, math, diffusion-models, evidence-lower-bound
  
  > **Comments:**
  > **username_1**: Where you see this derivation in your linked website? which equation number?  
  > **username_0**: 37 in the show me the proof part  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-11-25. Score: 0 | Accepted answer
  
  Indeed this sounds very confusing until you expand the product $\Pi$ terms as outlined below:
  $$\frac{p\_{\theta}(x\_1|x\_2)q(x\_1|x\_0)p\_{\theta}(x\_2|x\_3)q(x\_2|x\_0)...p\_{\theta}(x\_{T-1}|x\_T)q(x\_{T-1}|x\_0)}{q(x\_1|x\_2,x\_0)q(x\_2|x\_0)...q(x\_{T-2}|x\_{T-1},x\_0)q(x\_{T-1}|x\_0)q(x\_{T-1}|x\_T,x\_0)q(x\_T|x\_0)}$$
  
  Therefore now it's obvious that the cancelation is from all alternating $q$ terms with only one conditioned variable $x\_0$ in the numerator and denominator, respectively, *except* the first and last terms $\frac{q(x\_1|x\_0)}{q(x\_T|x\_0)}$ still remain. That's why in your reference the next step in show me proof section there appears one additional middle term $+\log \frac{q(x\_1|x\_0)}{q(x\_T|x\_0)}$. The author perhaps should expand like here to help reader's clear comprehension considering he's already written so many detailed steps.
  
  
  ---
  
  