Solved by ptrblck in post #2 
              
              
                Your code should work and both models as well as optimizers would be independent. 
However, if you don’t have any randomness in the models and are also feeding the same data as well as target into them, the training might be equal (unless non-deterministic operations are used internally with an unde…