Yeah, like I said, seeding is a bit tricky in such a situation. That is why in my example I’ve loaded the state_dict and also set the random seed for both forward passes to get the same result.
As @musicpiece explained, each additional call to the pseudo random number generator might yield a different result.
Since you are initializing an additional layer in your second model, the PRNG will get an additional call and the next random operation might behave differently (e.g. the Dropout layer might behave differently in both models). Also, if you call them sequentially like in my example without reseeding, Dropout will most likely behave differently.