ArneNx:

So as long as I want to perfectly reproduce the initial weights it’s safest to construct a new model or re-load the initial state-dict, right?


Yes, I would recommend to set the seed once in a script, initialize the model, and store the state_dict.
This state_dict is then the reference for this particular seed.
Making sure that all operations are executed in the same order is often too complicated.