Yeah, that sounds like a great idea; however, I’m getting this weird behavior now where after I did the following:
m1 = Model().to(device)
m2 = Model().to(device)
m2.load_state_dict(m1.state_dict())

And then manually checked whether the initial state of the parameters in both models was equivalent (which it was), the initial loss value was still different (and throughout)! Not only that, but the Accuracy (Automatic) went to 88.21% from 88.0%. Somehow loading m1's state dictionary into m2 affected m1?
I even tried the following:
m1 = Model().to(device)
m2 = deepcopy(m1)

Yet again, manually checking all the parameters showed they were indeed equivalent throughout the whole network. However, I’m still getting different results (though the Accuracy (Automatic) this time matches the original one (i.e., 88.0%)).