Answers.Body: <p>I don't see why it should not work.</p>
  <p>Though, the tough part is how to implement this.  The algorithm requires to sample a singular time point along the timestep sequence, draw <span class="math-container">$\epsilon$</span>, calculate <span class="math-container">$x_t$</span> for a batch, and evaluate the loss.</p>
  <p>In your case you should also create the final image for the whole batch at each individual evaluation of the loss, which is extremely time consuming.  But, other than that, I think it should work.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-05T18:33:44.367Z
Answers.Text: It's time consuming because I'm gonna have to do the whole denoising process? So, for example, it may include 50 denoising steps instead of the single step that you usually do during training, that is what you mean?
  
  Perhaps it would make sense to denoise using a single step only? then it would spare the full denoising process. Anyhow, I intend to work with low resolution (MNIST) images, so I'm not that worried about efficiency
Answers.UserDisplayName: Gilad Deutsch
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-06T11:46:43.640Z
Answers.Text: @GiladDeutsch yes this is what I meant with "time consuming".  You may try to partially mitigate the problem by skipping a few denoising steps (look up the reply by Ciodar to [this question](https://ai.stackexchange.com/questions/41068/about-cosine-noise-schedule-in-diffusion-model/41144#41144)), or by adding the penalization term at random only every _N_ batches.
  Keep me posted, I am curious to know about the solution you achieve.
Answers.UserDisplayName: Peblo
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-27T09:11:49.060Z
Answers.Text: are you sure that the fact that the diffusion model is recursive (i.e iteratively denoises the image) does not create an issue with this idea of learning using a loss on the created image? seems like this would create the same problems that RNNs have
Answers.UserDisplayName: Gilad Deutsch
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-31T12:51:35.113Z
Answers.Text: I am not sure which RNN issue you refer to, however -- regarding the recursiveness: keep in mind that the _reproduction_, i.e., the whole inverse process at prediction time, _is_ indeed recursive, but the learning phase is _not_.  During the learning phase the NN only needs to know the timestep $t$ under current evaluation, and upgrade the weights, it does _not_ need to recur.  Look at Algorithm 1 in Ho et al. 2020, step 5.
Answers.UserDisplayName: Peblo
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-04T15:49:27.563Z
Answers.LastActivityDate: 2023-07-04T15:49:27.563Z
Answers.UserDisplayName: Peblo
Body: <p>I would like to train a diffusion model with an additional loss on the created image. Without getting into too much details my intention is to do something like regularization, for example you may think that I want to make sure the created image is smooth, or something of the sort.
  My thinking was to add an additional loss during training, if the vanilla training process is:
  <span class="math-container">$$L = ||\epsilon-\epsilon_\theta(x_t, t)||$$</span>
  where <span class="math-container">$\epsilon_\theta$</span> is the model learning to predict the noise <span class="math-container">$\epsilon$</span> added to the original image. My suggested loss is:
  <span class="math-container">$$L = ||\epsilon-\epsilon_\theta(x_t, t)|| + \lambda * L'(img_\theta)$$</span>
  where <span class="math-container">$L'$</span> is my additional loss (which may for example induce smoothness or whatever), <span class="math-container">$img_\theta$</span> is the image that we get after denoising <span class="math-container">$x_t$</span> using the predicted noise.
  For standart models it is trivial that this makes sense. Due to the iterative nature of diffusion models I'm not sure if specifically for them it makes sense. I wasn't able to find any work that does something like this, would appreciate any help, does my additional loss makes sense to add?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-30T16:28:04.447Z
Comments.Text: Be careful not to mix different parameterizations: Here the first term is performing $\epsilon$-prediction (i.e you predict the noise part of the image), while your second term $img_\theta$, seems to be depending on a prediction of the denoised image. As @Peblo pointed out, training and sampling algorithms are typically different, and in your proposed loss you'd likely not be able to exploit the $\epsilon$-prediction reparameterization
Comments.UserDisplayName: Ciodar
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-06-18T07:55:41.970Z
Comments.Text: I have nearly the same question, do you have tried it? I think the key issue is the physical meaning during the intermediate steps, do the loss work when the generated thing is still a kind of noise?
Comments.UserDisplayName: Zero L
Tags: <diffusion-models>
Title: How to train Diffusion model with additional loss?
UserDisplayName: Gilad Deutsch
ThreadText: # How to train Diffusion model with additional loss?
  
  (Asked by: username_0 on 2023-06-25)
  
  I would like to train a diffusion model with an additional loss on the created image. Without getting into too much details my intention is to do something like regularization, for example you may think that I want to make sure the created image is smooth, or something of the sort.
  My thinking was to add an additional loss during training, if the vanilla training process is:
  $$L = ||\epsilon-\epsilon\_\theta(x\_t, t)||$$
  where $\epsilon\_\theta$ is the model learning to predict the noise $\epsilon$ added to the original image. My suggested loss is:
  $$L = ||\epsilon-\epsilon\_\theta(x\_t, t)|| + \lambda \* L'(img\_\theta)$$
  where $L'$ is my additional loss (which may for example induce smoothness or whatever), $img\_\theta$ is the image that we get after denoising $x\_t$ using the predicted noise.
  For standart models it is trivial that this makes sense. Due to the iterative nature of diffusion models I'm not sure if specifically for them it makes sense. I wasn't able to find any work that does something like this, would appreciate any help, does my additional loss makes sense to add?
  
  **Tags:** diffusion-models
  
  > **Comments:**
  > **username_1**: Be careful not to mix different parameterizations: Here the first term is performing $\epsilon$-prediction (i.e you predict the noise part of the image), while your second term $img_\theta$, seems to be depending on a prediction of the denoised image. As @username_3 pointed out, training and sampling algorithms are typically different, and in your proposed loss you'd likely not be able to exploit the $\epsilon$-prediction reparameterization  
  > **username_2**: I have nearly the same question, do you have tried it? I think the key issue is the physical meaning during the intermediate steps, do the loss work when the generated thing is still a kind of noise?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2023-07-04. Score: 0
  
  I don't see why it should not work.
  
  Though, the tough part is how to implement this. The algorithm requires to sample a singular time point along the timestep sequence, draw $\epsilon$, calculate $x\_t$ for a batch, and evaluate the loss.
  
  In your case you should also create the final image for the whole batch at each individual evaluation of the loss, which is extremely time consuming. But, other than that, I think it should work.
  
  > **Comments:**
  > **username_0**: It's time consuming because I'm gonna have to do the whole denoising process? So, for example, it may include 50 denoising steps instead of the single step that you usually do during training, that is what you mean?  Perhaps it would make sense to denoise using a single step only? then it would spare the full denoising process. Anyhow, I intend to work with low resolution (MNIST) images, so I'm not that worried about efficiency  
  > **username_3**: @username_0 yes this is what I meant with "time consuming".  You may try to partially mitigate the problem by skipping a few denoising steps (look up the reply by username_1 to [this question](https://ai.stackexchange.com/questions/41068/about-cosine-noise-schedule-in-diffusion-model/41144#41144)), or by adding the penalization term at random only every _N_ batches. Keep me posted, I am curious to know about the solution you achieve.  
  > **username_0**: are you sure that the fact that the diffusion model is recursive (i.e iteratively denoises the image) does not create an issue with this idea of learning using a loss on the created image? seems like this would create the same problems that RNNs have  
  > **username_3**: I am not sure which RNN issue you refer to, however -- regarding the recursiveness: keep in mind that the _reproduction_, i.e., the whole inverse process at prediction time, _is_ indeed recursive, but the learning phase is _not_.  During the learning phase the NN only needs to know the timestep $t$ under current evaluation, and upgrade the weights, it does _not_ need to recur.  Look at Algorithm 1 in Ho et al. 2020, step 5.  
  
  ---
  
  