Answers.Body: <p>This is (mostly) because of random weight initialization, each time you instance your model, starting weights are different, and during training the model weights converge to a different local minima.</p>
  <p>If you want reproducible results, you should fix the starting weights before training to a fixed value, this is typically done by setting a random seed.</p>
  <p>Also note that there are other components that can have stochastic behavior, like dataset shuffling, or giving random inputs to your diffusion model.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-12-01T09:21:27.067Z
Answers.Text: not only weights initialization, but also every other random seed utilized in other classes (like data shuffling, dropout and so on and so forth)
Answers.UserDisplayName: Edoardo Guerriero
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-12-01T10:09:43.337Z
Answers.Text: But the margin is very far. I thought it is related to diffusion model itself?
Answers.UserDisplayName: ProEns08
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-12-01T10:10:45.800Z
Answers.Text: I did not fix the seed for other trainings and found that the stability is better
Answers.UserDisplayName: ProEns08
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-12-01T10:12:24.280Z
Answers.Text: @ProEns08 Stability can have a lot of meanings in this context, I think you also need to define that. There are many variables that introduce randomness into the training process.
Answers.UserDisplayName: Dr. Snoopy
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-12-01T10:31:15.907Z
Answers.Text: @EdoardoGuerriero MC-Dropout can affect, but not Dropout, since it is disabled at test time. Other points I agree and I updated my answer.
Answers.UserDisplayName: Dr. Snoopy
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-12-01T08:36:58.587Z
Answers.LastActivityDate: 2022-12-01T10:22:17.097Z
Answers.LastEditDate: 2022-12-01T10:22:17.097Z
Answers.UserDisplayName: Dr. Snoopy
Body: <p>I ran an experiment using a diffusion model (SR3 model) and got good results (experiment using jupyter notebook). Then, after running the same notebook many times with the same configuration, I did not get the same results.</p>
  <p>Is there any explanation for this? Why I did not get the same result for the diffusion model by running the same notebook with the same configurations?</p>
  
Tags: <neural-networks><deep-learning><python><generative-model><diffusion-models>
Title: Is Diffusion model instable during the training?
UserDisplayName: ProEns08
ThreadText: # Is Diffusion model instable during the training?
  
  (Asked by: username_0 on 2022-12-01)
  
  I ran an experiment using a diffusion model (SR3 model) and got good results (experiment using jupyter notebook). Then, after running the same notebook many times with the same configuration, I did not get the same results.
  
  Is there any explanation for this? Why I did not get the same result for the diffusion model by running the same notebook with the same configurations?
  
  **Tags:** neural-networks, deep-learning, python, generative-model, diffusion-models
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-12-01. Score: 3 | Accepted answer
  
  This is (mostly) because of random weight initialization, each time you instance your model, starting weights are different, and during training the model weights converge to a different local minima.
  
  If you want reproducible results, you should fix the starting weights before training to a fixed value, this is typically done by setting a random seed.
  
  Also note that there are other components that can have stochastic behavior, like dataset shuffling, or giving random inputs to your diffusion model.
  
  > **Comments:**
  > **username_2**: not only weights initialization, but also every other random seed utilized in other classes (like data shuffling, dropout and so on and so forth)  
  > **username_0**: But the margin is very far. I thought it is related to diffusion model itself?  
  > **username_0**: I did not fix the seed for other trainings and found that the stability is better  
  > **username_1**: @ProEns08 Stability can have a lot of meanings in this context, I think you also need to define that. There are many variables that introduce randomness into the training process.  
  > **username_1**: @EdoardoGuerriero MC-Dropout can affect, but not Dropout, since it is disabled at test time. Other points I agree and I updated my answer.  
  
  ---
  
  