<p>This is (mostly) because of random weight initialization, each time you instance your model, starting weights are different, and during training the model weights converge to a different local minima.</p>
<p>If you want reproducible results, you should fix the starting weights before training to a fixed value, this is typically done by setting a random seed.</p>
<p>Also note that there are other components that can have stochastic behavior, like dataset shuffling, or giving random inputs to your diffusion model.</p>

CC BY-SA 4.0
2022-12-01T09:21:27.067Z
not only weights initialization, but also every other random seed utilized in other classes (like data shuffling, dropout and so on and so forth)
Edoardo Guerriero
CC BY-SA 4.0
2022-12-01T10:09:43.337Z
But the margin is very far. I thought it is related to diffusion model itself?
ProEns08
CC BY-SA 4.0
2022-12-01T10:10:45.800Z
I did not fix the seed for other trainings and found that the stability is better
ProEns08
CC BY-SA 4.0
2022-12-01T10:12:24.280Z
@ProEns08 Stability can have a lot of meanings in this context, I think you also need to define that. There are many variables that introduce randomness into the training process.
Dr. Snoopy
CC BY-SA 4.0
2022-12-01T10:31:15.907Z
@EdoardoGuerriero MC-Dropout can affect, but not Dropout, since it is disabled at test time. Other points I agree and I updated my answer.
Dr. Snoopy
CC BY-SA 4.0
2022-12-01T08:36:58.587Z
2022-12-01T10:22:17.097Z
2022-12-01T10:22:17.097Z
Dr. Snoopy
<p>I ran an experiment using a diffusion model (SR3 model) and got good results (experiment using jupyter notebook). Then, after running the same notebook many times with the same configuration, I did not get the same results.</p>
<p>Is there any explanation for this? Why I did not get the same result for the diffusion model by running the same notebook with the same configurations?</p>

<neural-networks><deep-learning><python><generative-model><diffusion-models>
Is Diffusion model instable during the training?
ProEns08
# Is Diffusion model instable during the training?

(Asked by: username_0 on 2022-12-01)

I ran an experiment using a diffusion model (SR3 model) and got good results (experiment using jupyter notebook). Then, after running the same notebook many times with the same configuration, I did not get the same results.

Is there any explanation for this? Why I did not get the same result for the diffusion model by running the same notebook with the same configurations?

**Tags:** neural-networks, deep-learning, python, generative-model, diffusion-models


---

## 1 answers

### Answer by username_1 on 2022-12-01. Score: 3 | Accepted answer

This is (mostly) because of random weight initialization, each time you instance your model, starting weights are different, and during training the model weights converge to a different local minima.

If you want reproducible results, you should fix the starting weights before training to a fixed value, this is typically done by setting a random seed.

Also note that there are other components that can have stochastic behavior, like dataset shuffling, or giving random inputs to your diffusion model.

> **Comments:**
> **username_2**: not only weights initialization, but also every other random seed utilized in other classes (like data shuffling, dropout and so on and so forth)  
> **username_0**: But the margin is very far. I thought it is related to diffusion model itself?  
> **username_0**: I did not fix the seed for other trainings and found that the stability is better  
> **username_1**: @username_0 Stability can have a lot of meanings in this context, I think you also need to define that. There are many variables that introduce randomness into the training process.  
> **username_1**: @EdoardoGuerriero MC-Dropout can affect, but not Dropout, since it is disabled at test time. Other points I agree and I updated my answer.  

---

