Hello,
I have two models that are supposed to be copies of each other, but perform differently. Look at this:
>>> repr(model1) == repr(model2) #have the same structure
True
>>> for idx, (p1, p2) in enumerate(zip(model1.named_parameters(), model2.named_parameters())):
    if not p1[0] == p2[0]:
        print('different parameter order for idx {}'.format(idx))
    if not torch.equal(p1[1].data, p2[1].data):
        print('idx {} not equal'.format(idx))
#nothing is printed, means they are the same
>>> evaluateModel(model1, test_loader, fastEvaluation=False)
0.8836
>>> evaluateModel(model2, test_loader, fastEvaluation=False)
0.8735

What could be the problem? They are instances of the same class, created with the same parameters. Have the same structure, and the only thing I do is changing the weights in a certain way for model1, then inverting these changes. Point being, the two models have the same weight, I expect them to perform identically.
Why does this not happen? What could be the problem? What other fields should I check to make sure that the models are the same?
P.S. note that evaluateModel automatically calls eval() on the input model, so this can’t be a train vs eval mode difference.