You could either load the state_dict from one model to the other or seed the code carefully before initializing the models and make sure the order of calls into pseudorandom number generator is the same.