kuonlp:

I would expect that when I do the output = model(im) autograd realizes that the shapes have changed already and should ignore whatever it knows from Iteration 1.


That’s not the case, since you are manipulating the underlying .data attributes, which Autograd doesn’t track, and which is thus not recommended.
Try to assign the new parameters directly and, if needed, wrap it into a with torch.no_grad() guard.