Answers.Body: <p>Question 1 is indeed correct. From there, the integral for <span class="math-container">$\int dx^{(t)} q(x^{(t)}|x^{(t-1)})$</span> is also correct. The problem is that this expression does not appear in your intended integral.
  In
  <span class="math-container">$$\int dx^{(0)} q(x^{(0)}) \Bigg(\prod_{t=1}^{T-1} \int dx^{(t)}q(x^{(t)}|x^{(t-1)}) \Bigg) \int dx^{(T)}q(x^{(T)}|x^{(T-1)}) \text{ log } p(x^{(T)})$$</span>
  if you want to integrate out the <span class="math-container">$x^{(t)}$</span> variable you actually need to simplify:
  <span class="math-container">$$ \int dx^{(t)} q(x^{(t)}|x^{(t-1)}) q(x^{(t+1)}|x^{(t)}) $$</span>
  for <span class="math-container">$t \notin {0, T}$</span>. For a Markov chain this is doable, using the Chapman–Kolmogorov equation, which implies:
  <span class="math-container">$$ \int dx^{(t)} q(x^{(t)}|x^{(t-1)}) q(x^{(t+1)}|x^{(t)})  = q(x^{(t+1)}|x^{(t-1)}) $$</span></p>
  <p>Doing the above for every single variable except <span class="math-container">$x^{(T)}$</span> and <span class="math-container">$x^{(0)}$</span> leads to:
  <span class="math-container">$$\int dx^{(T)} \int dx^{(0)} q(x^{(0)}) q(x^{(T)}|x^{(0)}) \log p(x^{(T)}) = \int dx^{(T)} q(x^{(T)}) \log p(x^{(T)})$$</span>
  where in the end I integrated on <span class="math-container">$x^{(0)}$</span> and obtained the term you wanted.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-03T21:32:31.133Z
Answers.Text: Thank you, ThunderSmotch. This was helpful! :)
Answers.UserDisplayName: ahxmeds
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-01T08:57:02.107Z
Answers.LastActivityDate: 2024-04-01T08:57:02.107Z
Answers.UserDisplayName: ThunderSmotch
Body: <p>I have recently been learning about diffusion models and trying to derive all the results in the paper by Sohl-Dickstein, et. al, &quot;Deep Unsupervised Learning using Nonequilibrium Thermodynamics&quot; (2015): <a href="https://arxiv.org/pdf/1503.03585.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1503.03585.pdf</a></p>
  <p>In the Appendix B of this paper, they define the log-likelihood lower bound term as, <span class="math-container">$$K = \int{dx^{(0...T)}} q(x^{(0...T)}) \text{log} \Bigg[p(x^{(T)}) \prod_{t=1}^{T} \frac{p(x^{(t-1)}|x^{(t)})}{q(x^{(t)}|x^{(t-1)})} \Bigg]$$</span>.</p>
  <p>They further try to isolate the <span class="math-container">$p(x^{(T)})$</span> in the square bracket. The next step of the calculation should be,</p>
  <p><span class="math-container">$$K = \int{dx^{(0...T)}} q(x^{(0...T)}) \text{log} \sum_{t=1}^{T}\Bigg[ \frac{p(x^{(t-1)}|x^{(t)})}{q(x^{(t)}|x^{(t-1)})} \Bigg] + \int{dx^{(0...T)}} q(x^{(0...T)}) \text{ log } p(x^{(T)})$$</span></p>
  <p>The first integral above looks the same as the first term in the paper, I am trying to make the second integral above go to <span class="math-container">$$\int dx^{(T)} q(x^{(T)}) \text{ log } p(x^{(T)})$$</span></p>
  <p>For this, I am using the fact that <span class="math-container">$q(x^{(0...T)})$</span> can be written as,</p>
  <p><span class="math-container">$$q(x^{(0...T)}) = q(x^{(0)}) \prod_{t=1}^{T} q(x^{(t)}|x^{(t-1)})$$</span></p>
  <p>Then I am writing the second integral above as,</p>
  <p><span class="math-container">$$\int{dx^{(0...T)}} q(x^{(0...T)}) \text{ log } p(x^{(T)}) \\= \int dx^{(0)} q(x^{(0)}) \int dx^{(1)}q(x^{(1)}|x^{(0)}) \ldots \int dx^{(T-1)}q(x^{(T-1)}|x^{(T-2)}) \int dx^{(T)}q(x^{(T)}|x^{(T-1)}) \text{ log } p(x^{(T)}) \\ = \int dx^{(0)} q(x^{(0)}) \Bigg(\prod_{t=1}^{T-1} \int dx^{(t)}q(x^{(t)}|x^{(t-1)}) \Bigg) \int dx^{(T)}q(x^{(T)}|x^{(T-1)}) \text{ log } p(x^{(T)})$$</span></p>
  <p><strong>Question 1: Is the above expression correct?</strong></p>
  <p>Moreover, I think we can also express any <span class="math-container">$q(x^{(t)}|x^{(t-1)}) = \frac{q(x^{(t)}, x^{(t-1)})}{q(x^{(t-1)})}$</span> (using conditional probability rule). Does that mean that each of the integrals within the parenthesis in the previous step can be written as,</p>
  <p><span class="math-container">$$ \int dx^{(t)}q(x^{(t)}|x^{(t-1)}) = \int dx^{(t)} \frac{q(x^{(t)}, x^{(t-1)})}{q(x^{(t-1)})} = \frac{1}{q(x^{(t-1)})} \int dx^{(t)} q(x^{(t)}, x^{(t-1)}) = \frac{1}{q(x^{(t-1)})} q(x^{(t-1)}) = 1$$</span></p>
  <p>Also, we have <span class="math-container">$\int{dx^{(0)}} q(x^{(0)}) = 1$</span>. Following the above line of thought, the second integral becomes,</p>
  <p><span class="math-container">$$\int dx^{(T)}q(x^{(T)}|x^{(T-1)}) \text{ log } p(x^{(T)})$$</span></p>
  <p>I am not able to proceed further than this step.</p>
  <p><strong>Question 2: Can someone point out if there is a mistake in one of my previous steps and how to proceed with getting the required expression for the second integral term of <span class="math-container">$K$</span>?</strong></p>
  
Tags: <generative-model><variational-autoencoder><probability-distribution><diffusion-models><conditional-probability>
Title: Trying to understand some derivation in the paper: Deep Unsupervised Learning using Nonequilibrium Thermodynamics
UserDisplayName: ahxmeds
ThreadText: # Trying to understand some derivation in the paper: Deep Unsupervised Learning using Nonequilibrium Thermodynamics
  
  (Asked by: username_0 on 2024-03-28)
  
  I have recently been learning about diffusion models and trying to derive all the results in the paper by Sohl-Dickstein, et. al, "Deep Unsupervised Learning using Nonequilibrium Thermodynamics" (2015): <https://arxiv.org/pdf/1503.03585.pdf>
  
  In the Appendix B of this paper, they define the log-likelihood lower bound term as, $$K = \int{dx^{(0...T)}} q(x^{(0...T)}) \text{log} \Bigg[p(x^{(T)}) \prod\_{t=1}^{T} \frac{p(x^{(t-1)}|x^{(t)})}{q(x^{(t)}|x^{(t-1)})} \Bigg]$$.
  
  They further try to isolate the $p(x^{(T)})$ in the square bracket. The next step of the calculation should be,
  
  $$K = \int{dx^{(0...T)}} q(x^{(0...T)}) \text{log} \sum\_{t=1}^{T}\Bigg[ \frac{p(x^{(t-1)}|x^{(t)})}{q(x^{(t)}|x^{(t-1)})} \Bigg] + \int{dx^{(0...T)}} q(x^{(0...T)}) \text{ log } p(x^{(T)})$$
  
  The first integral above looks the same as the first term in the paper, I am trying to make the second integral above go to $$\int dx^{(T)} q(x^{(T)}) \text{ log } p(x^{(T)})$$
  
  For this, I am using the fact that $q(x^{(0...T)})$ can be written as,
  
  $$q(x^{(0...T)}) = q(x^{(0)}) \prod\_{t=1}^{T} q(x^{(t)}|x^{(t-1)})$$
  
  Then I am writing the second integral above as,
  
  $$\int{dx^{(0...T)}} q(x^{(0...T)}) \text{ log } p(x^{(T)}) \\= \int dx^{(0)} q(x^{(0)}) \int dx^{(1)}q(x^{(1)}|x^{(0)}) \ldots \int dx^{(T-1)}q(x^{(T-1)}|x^{(T-2)}) \int dx^{(T)}q(x^{(T)}|x^{(T-1)}) \text{ log } p(x^{(T)}) \\ = \int dx^{(0)} q(x^{(0)}) \Bigg(\prod\_{t=1}^{T-1} \int dx^{(t)}q(x^{(t)}|x^{(t-1)}) \Bigg) \int dx^{(T)}q(x^{(T)}|x^{(T-1)}) \text{ log } p(x^{(T)})$$
  
  **Question 1: Is the above expression correct?**
  
  Moreover, I think we can also express any $q(x^{(t)}|x^{(t-1)}) = \frac{q(x^{(t)}, x^{(t-1)})}{q(x^{(t-1)})}$ (using conditional probability rule). Does that mean that each of the integrals within the parenthesis in the previous step can be written as,
  
  $$ \int dx^{(t)}q(x^{(t)}|x^{(t-1)}) = \int dx^{(t)} \frac{q(x^{(t)}, x^{(t-1)})}{q(x^{(t-1)})} = \frac{1}{q(x^{(t-1)})} \int dx^{(t)} q(x^{(t)}, x^{(t-1)}) = \frac{1}{q(x^{(t-1)})} q(x^{(t-1)}) = 1$$
  
  Also, we have $\int{dx^{(0)}} q(x^{(0)}) = 1$. Following the above line of thought, the second integral becomes,
  
  $$\int dx^{(T)}q(x^{(T)}|x^{(T-1)}) \text{ log } p(x^{(T)})$$
  
  I am not able to proceed further than this step.
  
  **Question 2: Can someone point out if there is a mistake in one of my previous steps and how to proceed with getting the required expression for the second integral term of $K$?**
  
  **Tags:** generative-model, variational-autoencoder, probability-distribution, diffusion-models, conditional-probability
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-04-01. Score: 0
  
  Question 1 is indeed correct. From there, the integral for $\int dx^{(t)} q(x^{(t)}|x^{(t-1)})$ is also correct. The problem is that this expression does not appear in your intended integral.
  In
  $$\int dx^{(0)} q(x^{(0)}) \Bigg(\prod\_{t=1}^{T-1} \int dx^{(t)}q(x^{(t)}|x^{(t-1)}) \Bigg) \int dx^{(T)}q(x^{(T)}|x^{(T-1)}) \text{ log } p(x^{(T)})$$
  if you want to integrate out the $x^{(t)}$ variable you actually need to simplify:
  $$ \int dx^{(t)} q(x^{(t)}|x^{(t-1)}) q(x^{(t+1)}|x^{(t)}) $$
  for $t \notin {0, T}$. For a Markov chain this is doable, using the Chapman–Kolmogorov equation, which implies:
  $$ \int dx^{(t)} q(x^{(t)}|x^{(t-1)}) q(x^{(t+1)}|x^{(t)}) = q(x^{(t+1)}|x^{(t-1)}) $$
  
  Doing the above for every single variable except $x^{(T)}$ and $x^{(0)}$ leads to:
  $$\int dx^{(T)} \int dx^{(0)} q(x^{(0)}) q(x^{(T)}|x^{(0)}) \log p(x^{(T)}) = \int dx^{(T)} q(x^{(T)}) \log p(x^{(T)})$$
  where in the end I integrated on $x^{(0)}$ and obtained the term you wanted.
  
  > **Comments:**
  > **username_0**: Thank you, username_1. This was helpful! :)  
  
  ---
  
  