Answers.Body: <p>ELBO derivation for DDPM doesn't need the convoluted Chapman-Kolmogorov property for the Markov process, your tutorial also doesn't mention it at all. They key difficulty is to reverse the direction of <span class="math-container">$q_{\phi}(x_t|x_{t-1})$</span> in the original consistency term of the ELBO via Bayes rule and one more conditional rv <span class="math-container">$x_0$</span>. Then the consistency term just takes expectation wrt to the tractable distribution <span class="math-container">$q_{\phi}(x_t|x_0)$</span> instead of the original odd <span class="math-container">$q_{\phi}(x_{t-1},x_{t+1}|x_0)$</span>. Further on you can use the nice property of Gaussians (every type of conditional distribution here is Gaussian) of the forward and reverse consistent Markov chains to get a simple closed form training objective.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-22T02:53:13.583Z
Answers.LastActivityDate: 2024-07-22T02:58:25.403Z
Answers.LastEditDate: 2024-07-22T02:58:25.403Z
Answers.UserDisplayName: cinch
Body: <p>I have been studying diffusion models from this tutorial: <a href="https://arxiv.org/abs/2403.18103" rel="nofollow noreferrer">https://arxiv.org/abs/2403.18103</a> and trying to derive all results as I read it. Although this tutorial is very comprehensive, it skips many of the derivation steps. I am currently stuck in the derivation of the Evidence Lower Bound (ELBO) terms for variational diffusion model (Page 17). The ELBO consists of 3 terms: reconstruction loss term, prior matching and consistency term. I am struggling with the derivation for the consistency term as I am trying to expand the expectation as an integral over the specified probability distribution <span class="math-container">$q_{\phi}(x_{1:T}|x_0)$</span>. To simplify the expression, I am using the Chapman-Kolmogorov property for the Markov process, as explained <a href="https://ai.stackexchange.com/questions/45272/trying-to-understand-some-derivation-in-the-paper-deep-unsupervised-learning-us">here</a></p>
  <p><span class="math-container">$$\int dx_t q_{\phi}(x_{t+1}|x_t)q_{\phi}(x_t|x_{t-1}) = q_{\phi}(x_{t+1}|x_{t-1})$$</span></p>
  <p>Solving the consistency term,</p>
  <p><span class="math-container">$$E_{q_{\phi}(x_{1:T}|x_0)}\Big[\text{log} \prod_{t=1}^{T-1} \frac{p(x_t|x_{t+1})}{q_{\phi}(x_{t}|x_{t-1})}\Big] = \sum_{t=1}^{T-1} E_{q_{\phi}(x_{1:T}|x_0)}\Big[\text{log} \frac{p(x_t|x_{t+1})}{q_{\phi}(x_{t}|x_{t-1})} \Big] = \sum_{t=1}^{T-1} \int dx_1 dx_2 \ldots dx_{t-2}dx_{t-1}dx_{t}dx_{t+1}dx_{t+2} \ldots dx_{T} q_{\phi}(x_{T}|x_{T-1}) \ldots q_{\phi}(x_{t+2}|x_{t+1}) q_{\phi}(x_{t+1}|x_{t}) q_{\phi}(x_{t}|x_{t-1}) q_{\phi}(x_{t-1}|x_{t-2}) q_{\phi}(x_{t-2}|x_{t-3}) \ldots q_{\phi}(x_{1}|x_{0}) \text{log} \frac{p(x_t|x_{t+1})}{q_{\phi}(x_t | x_{t-1})}$$</span></p>
  <p>Using the Chapman-Kolmogorov property, I can perform the integral for the indices <span class="math-container">$t \in$</span> {<span class="math-container">$1, \ldots, (t-2)$</span>} <span class="math-container">$\cup$</span> { <span class="math-container">$(t+2),\ldots, (T-1)$</span>}. After these integrations, the expression becomes,</p>
  <p><span class="math-container">$$
  \sum_{t=1}^{T-1} \int dx_{t-1} dx_{t} dx_{t+1} dx_{T} q_{\phi}(x_T | x_{t+1}) q_{\phi}(x_{t+1}|x_{t}) q_{\phi}(x_{t}|x_{t-1}) q_{\phi}(x_{t-1}|x_{0}) \text{log} \frac{p(x_t|x_{t+1})}{q_{\phi}(x_t | x_{t-1})} 
  $$</span></p>
  <p><strong>Question 1:</strong> In the above expression, can I perform the integral over the index <span class="math-container">$T$</span>, as in <span class="math-container">$\int dx_T q_{\phi}(x_T|x_{t+1}) = 1$</span>? Is this correct?</p>
  <p>Let's say, I did perform the integral over index <span class="math-container">$T$</span>, and proceed further as,</p>
  <p><span class="math-container">$$
  \sum_{t=1}^{T-1} \int dx_{t-1} dx_{t} dx_{t+1} q_{\phi}(x_{t+1}|x_{t}) q_{\phi}(x_{t}|x_{t-1}) q_{\phi}(x_{t-1}|x_{0}) \text{log} \frac{p(x_t|x_{t+1})}{q_{\phi}(x_t | x_{t-1})} 
  $$</span>
  <span class="math-container">$$
  = -\sum_{t=1}^{T-1} \int dx_{t-1} dx_{t+1} q_{\phi}(x_{t+1}|x_{t})  q_{\phi}(x_{t-1}|x_{0}) \int dx_{t}  q_{\phi}(x_{t}|x_{t-1}) \text{log} \frac{q_{\phi}(x_t | x_{t-1})}{p(x_t|x_{t+1})}
  $$</span>
  <span class="math-container">$$
  = -\sum_{t=1}^{T-1} \int dx_{t-1} dx_{t+1} q_{\phi}(x_{t+1}|x_{t})  q_{\phi}(x_{t-1}|x_{0}) D_{KL}(q_{\phi}(x_t | x_{t-1}) \Vert p(x_t | x_{t+1}))
  $$</span></p>
  <p><strong>Question 2:</strong> Is the above step correct?</p>
  <p>The final expression is supposed to look like the following, where the expectation is over <span class="math-container">$q_{\phi}(x_{t+1, t-1}|x_0)$</span> but I am not sure how to reduce my above expression to this:</p>
  <p><span class="math-container">$$
  -\sum_{t=1}^{T-1} E_{q_{\phi}(x_{t+1, t-1}|x_0)}\Big [ D_{KL}(q_{\phi}(x_t | x_{t-1}) \Vert p(x_t | x_{t+1})) \Big]
  $$</span></p>
  <p>Any feedback is appreciated. Thanks in advance!</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-23T09:24:51.773Z
Comments.Text: Please, ask only one question per post. You can always create multiple posts and reference other posts.
Comments.UserDisplayName: nbro
Tags: <variational-autoencoder><diffusion-models><evidence-lower-bound><conditional-probability>
Title: Derivation of the consistency term in the DDPM Evidence Lower Bound (ELBO)
UserDisplayName: ahxmeds
ThreadText: # Derivation of the consistency term in the DDPM Evidence Lower Bound (ELBO)
  
  (Asked by: username_0 on 2024-07-21)
  
  I have been studying diffusion models from this tutorial: <https://arxiv.org/abs/2403.18103> and trying to derive all results as I read it. Although this tutorial is very comprehensive, it skips many of the derivation steps. I am currently stuck in the derivation of the Evidence Lower Bound (ELBO) terms for variational diffusion model (Page 17). The ELBO consists of 3 terms: reconstruction loss term, prior matching and consistency term. I am struggling with the derivation for the consistency term as I am trying to expand the expectation as an integral over the specified probability distribution $q\_{\phi}(x\_{1:T}|x\_0)$. To simplify the expression, I am using the Chapman-Kolmogorov property for the Markov process, as explained [here](https://ai.stackexchange.com/questions/45272/trying-to-understand-some-derivation-in-the-paper-deep-unsupervised-learning-us)
  
  $$\int dx\_t q\_{\phi}(x\_{t+1}|x\_t)q\_{\phi}(x\_t|x\_{t-1}) = q\_{\phi}(x\_{t+1}|x\_{t-1})$$
  
  Solving the consistency term,
  
  $$E\_{q\_{\phi}(x\_{1:T}|x\_0)}\Big[\text{log} \prod\_{t=1}^{T-1} \frac{p(x\_t|x\_{t+1})}{q\_{\phi}(x\_{t}|x\_{t-1})}\Big] = \sum\_{t=1}^{T-1} E\_{q\_{\phi}(x\_{1:T}|x\_0)}\Big[\text{log} \frac{p(x\_t|x\_{t+1})}{q\_{\phi}(x\_{t}|x\_{t-1})} \Big] = \sum\_{t=1}^{T-1} \int dx\_1 dx\_2 \ldots dx\_{t-2}dx\_{t-1}dx\_{t}dx\_{t+1}dx\_{t+2} \ldots dx\_{T} q\_{\phi}(x\_{T}|x\_{T-1}) \ldots q\_{\phi}(x\_{t+2}|x\_{t+1}) q\_{\phi}(x\_{t+1}|x\_{t}) q\_{\phi}(x\_{t}|x\_{t-1}) q\_{\phi}(x\_{t-1}|x\_{t-2}) q\_{\phi}(x\_{t-2}|x\_{t-3}) \ldots q\_{\phi}(x\_{1}|x\_{0}) \text{log} \frac{p(x\_t|x\_{t+1})}{q\_{\phi}(x\_t | x\_{t-1})}$$
  
  Using the Chapman-Kolmogorov property, I can perform the integral for the indices $t \in$ {$1, \ldots, (t-2)$} $\cup$ { $(t+2),\ldots, (T-1)$}. After these integrations, the expression becomes,
  
  $$
  \sum\_{t=1}^{T-1} \int dx\_{t-1} dx\_{t} dx\_{t+1} dx\_{T} q\_{\phi}(x\_T | x\_{t+1}) q\_{\phi}(x\_{t+1}|x\_{t}) q\_{\phi}(x\_{t}|x\_{t-1}) q\_{\phi}(x\_{t-1}|x\_{0}) \text{log} \frac{p(x\_t|x\_{t+1})}{q\_{\phi}(x\_t | x\_{t-1})}
  $$
  
  **Question 1:** In the above expression, can I perform the integral over the index $T$, as in $\int dx\_T q\_{\phi}(x\_T|x\_{t+1}) = 1$? Is this correct?
  
  Let's say, I did perform the integral over index $T$, and proceed further as,
  
  $$
  \sum\_{t=1}^{T-1} \int dx\_{t-1} dx\_{t} dx\_{t+1} q\_{\phi}(x\_{t+1}|x\_{t}) q\_{\phi}(x\_{t}|x\_{t-1}) q\_{\phi}(x\_{t-1}|x\_{0}) \text{log} \frac{p(x\_t|x\_{t+1})}{q\_{\phi}(x\_t | x\_{t-1})}
  $$
  $$
  = -\sum\_{t=1}^{T-1} \int dx\_{t-1} dx\_{t+1} q\_{\phi}(x\_{t+1}|x\_{t}) q\_{\phi}(x\_{t-1}|x\_{0}) \int dx\_{t} q\_{\phi}(x\_{t}|x\_{t-1}) \text{log} \frac{q\_{\phi}(x\_t | x\_{t-1})}{p(x\_t|x\_{t+1})}
  $$
  $$
  = -\sum\_{t=1}^{T-1} \int dx\_{t-1} dx\_{t+1} q\_{\phi}(x\_{t+1}|x\_{t}) q\_{\phi}(x\_{t-1}|x\_{0}) D\_{KL}(q\_{\phi}(x\_t | x\_{t-1}) \Vert p(x\_t | x\_{t+1}))
  $$
  
  **Question 2:** Is the above step correct?
  
  The final expression is supposed to look like the following, where the expectation is over $q\_{\phi}(x\_{t+1, t-1}|x\_0)$ but I am not sure how to reduce my above expression to this:
  
  $$
  -\sum\_{t=1}^{T-1} E\_{q\_{\phi}(x\_{t+1, t-1}|x\_0)}\Big [ D\_{KL}(q\_{\phi}(x\_t | x\_{t-1}) \Vert p(x\_t | x\_{t+1})) \Big]
  $$
  
  Any feedback is appreciated. Thanks in advance!
  
  **Tags:** variational-autoencoder, diffusion-models, evidence-lower-bound, conditional-probability
  > **Comments:**
  > **username_1**: Please, ask only one question per post. You can always create multiple posts and reference other posts.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2024-07-22. Score: 0
  
  ELBO derivation for DDPM doesn't need the convoluted Chapman-Kolmogorov property for the Markov process, your tutorial also doesn't mention it at all. They key difficulty is to reverse the direction of $q\_{\phi}(x\_t|x\_{t-1})$ in the original consistency term of the ELBO via Bayes rule and one more conditional rv $x\_0$. Then the consistency term just takes expectation wrt to the tractable distribution $q\_{\phi}(x\_t|x\_0)$ instead of the original odd $q\_{\phi}(x\_{t-1},x\_{t+1}|x\_0)$. Further on you can use the nice property of Gaussians (every type of conditional distribution here is Gaussian) of the forward and reverse consistent Markov chains to get a simple closed form training objective.
  
  
  ---
  
  