Answers.Body: <p>The relationship between <span class="math-container">$x_t$</span> and <span class="math-container">$x_{t-1}$</span> is as follows:
  <span class="math-container">$$
  x_t = \sqrt{1-\beta_t}x_{t-1}+\sqrt{\beta_t}\epsilon_t,\quad \epsilon_t\sim\mathcal{N}(0,I).
  $$</span>
  Not only is a small amount of noise added, the original image is also scaled down slightly.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-11T02:00:49.227Z
Answers.LastActivityDate: 2023-01-11T02:00:49.227Z
Answers.UserDisplayName: Patrick Johnstone
Answers.Body: <p>The purpose of <span class="math-container">$q(x_t|x_{t-1})$</span> is: given that the random variable <span class="math-container">$x_{t-1}$</span> is sampled to be a specified value, what is the probability density function for <span class="math-container">$x_t$</span>? So we're using the <em>sampled</em> value of <span class="math-container">$x_{t-1}$</span> to calculate the mean of the probability density function, which is <span class="math-container">$\sqrt{1-\beta_t} x_{t-1}$</span>. We're <em>not</em> adding <span class="math-container">$x_{t-1}$</span> and <span class="math-container">$\epsilon_t$</span>. Instead, we are steering the mean towards zero a little bit each time step (because <span class="math-container">$\sqrt{1-\beta_t}&lt;1$</span>), while adding a little noise.</p>
  <p>To implement this, we execute the sampling of <span class="math-container">$x_t$</span> by taking the mean (the constant <span class="math-container">$\sqrt{1-\beta_t} x_{t-1}$</span>) and adding a sample from the standard normal distribution scaled by <span class="math-container">$\sqrt{\beta_t}$</span>.</p>
  <p>Of course, the probability chain rule then provides us the desired (nonconditional) pdf for <span class="math-container">$x_T$</span> at the final time.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-12T14:55:33.170Z
Answers.LastActivityDate: 2023-04-12T14:55:33.170Z
Answers.UserDisplayName: sunfishstanford
Answers.Body: <p>The image data <span class="math-container">$\mathbf{x}_{t-1}$</span> is not a constant <span class="math-container">$\mathbf{c}$</span>. It's itself a distribution. Different permutations of pixels have different probabilities.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-04T14:08:53.720Z
Answers.Text: Can you please expand this answer? How is the sqrt(1-beta_t) derived?
Answers.UserDisplayName: PascalIv
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-12-09T19:40:19.797Z
Answers.LastActivityDate: 2022-12-09T19:40:19.797Z
Answers.UserDisplayName: lkjsfkshd
Answers.Body: <p>Maybe <a href="https://www.youtube.com/watch?v=cS6JQpEY9cs" rel="nofollow noreferrer">this</a> video will help with the understanding.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-02T15:17:57.123Z
Answers.Text: Hi @sumit dugar and welcome to AI Stack Exchange! If possible, please include some details of the video in your answer. Thank you for posting, and we hope to see you again soon on this site!
Answers.UserDisplayName: DeepQZero
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-01T11:57:28.510Z
Answers.LastActivityDate: 2024-01-01T11:58:04.113Z
Answers.LastEditDate: 2024-01-01T11:58:04.113Z
Answers.UserDisplayName: sumit dugar
Body: <p>In papers and other material regarding diffusion models the forward diffusion process is defined by adding a small amount of Gaussian noise to an image <span class="math-container">$x_0$</span> for <span class="math-container">$T$</span> time steps. In each time step the noise has a variance of <span class="math-container">$\beta_t$</span>. This process produces a sequence of noisy samples: <span class="math-container">$x_1, x_2, x_3... x_T$</span> such that:
  <span class="math-container">$q(x_t|x_{t-1}) = N(x_t; \sqrt{1-\beta_t} x_{t-1}, \beta_tI)$</span></p>
  <p>I don't understand why this is <span class="math-container">$q(x_t|x_{t-1})$</span> distribution. When adding a constant <span class="math-container">$c$</span> to a normal random variable with mean <span class="math-container">$\mu$</span> and variance <span class="math-container">$\sigma^2$</span> we get a new random variable with the same variance and a mean of <span class="math-container">$c+\mu$</span>. Therefore, I expect <span class="math-container">$q(x_t|x_{t-1})$</span> to be: <span class="math-container">$q(x_t|x_{t-1}) = x_{t-1} + \epsilon_t = N(x_t; x_{t-1}, \beta_t I)$</span> where <span class="math-container">$\epsilon_t=N(\epsilon_t; 0, \beta_t  I)$</span></p>
  <p>Any help will be appreciated.</p>
  
Tags: <image-generation><markov-chain><diffusion-models>
Title: Forward Diffusion Process Derivation In Diffusion Models
UserDisplayName: nkam
ThreadText: # Forward Diffusion Process Derivation In Diffusion Models
  
  (Asked by: username_0 on 2022-11-23)
  
  In papers and other material regarding diffusion models the forward diffusion process is defined by adding a small amount of Gaussian noise to an image $x\_0$ for $T$ time steps. In each time step the noise has a variance of $\beta\_t$. This process produces a sequence of noisy samples: $x\_1, x\_2, x\_3... x\_T$ such that:
  $q(x\_t|x\_{t-1}) = N(x\_t; \sqrt{1-\beta\_t} x\_{t-1}, \beta\_tI)$
  
  I don't understand why this is $q(x\_t|x\_{t-1})$ distribution. When adding a constant $c$ to a normal random variable with mean $\mu$ and variance $\sigma^2$ we get a new random variable with the same variance and a mean of $c+\mu$. Therefore, I expect $q(x\_t|x\_{t-1})$ to be: $q(x\_t|x\_{t-1}) = x\_{t-1} + \epsilon\_t = N(x\_t; x\_{t-1}, \beta\_t I)$ where $\epsilon\_t=N(\epsilon\_t; 0, \beta\_t I)$
  
  Any help will be appreciated.
  
  **Tags:** image-generation, markov-chain, diffusion-models
  
  
  ---
  
  ## 4 answers
  
  ### Answer by username_1 on 2023-01-11. Score: 1 | Accepted answer
  
  The relationship between $x\_t$ and $x\_{t-1}$ is as follows:
  $$
  x\_t = \sqrt{1-\beta\_t}x\_{t-1}+\sqrt{\beta\_t}\epsilon\_t,\quad \epsilon\_t\sim\mathcal{N}(0,I).
  $$
  Not only is a small amount of noise added, the original image is also scaled down slightly.
  
  
  ---
  
  ### Answer by username_2 on 2023-04-12. Score: 0
  
  The purpose of $q(x\_t|x\_{t-1})$ is: given that the random variable $x\_{t-1}$ is sampled to be a specified value, what is the probability density function for $x\_t$? So we're using the *sampled* value of $x\_{t-1}$ to calculate the mean of the probability density function, which is $\sqrt{1-\beta\_t} x\_{t-1}$. We're *not* adding $x\_{t-1}$ and $\epsilon\_t$. Instead, we are steering the mean towards zero a little bit each time step (because $\sqrt{1-\beta\_t}<1$), while adding a little noise.
  
  To implement this, we execute the sampling of $x\_t$ by taking the mean (the constant $\sqrt{1-\beta\_t} x\_{t-1}$) and adding a sample from the standard normal distribution scaled by $\sqrt{\beta\_t}$.
  
  Of course, the probability chain rule then provides us the desired (nonconditional) pdf for $x\_T$ at the final time.
  
  
  ---
  
  ### Answer by username_3 on 2022-12-09. Score: 0
  
  The image data $\mathbf{x}\_{t-1}$ is not a constant $\mathbf{c}$. It's itself a distribution. Different permutations of pixels have different probabilities.
  
  > **Comments:**
  > **username_4**: Can you please expand this answer? How is the sqrt(1-beta_t) derived?  
  
  ---
  
  ### Answer by username_5 on 2024-01-01. Score: 0
  
  Maybe [this](https://www.youtube.com/watch?v=cS6JQpEY9cs) video will help with the understanding.
  
  > **Comments:**
  > **username_6**: Hi @username_5 and welcome to AI Stack Exchange! If possible, please include some details of the video in your answer. Thank you for posting, and we hope to see you again soon on this site!  
  
  ---
  
  