model2's parameters won’t be updated unless you call model2_opt.step().
However, if your model contains batchnorm layers, the running estimates will be updated during the forward pass, if you keep these layers in .train() mode.
Also, dropout will be applied, but of course there is nothing to update.
That being said, the .grad attributes of model2 will accumulate the gradients, if you don’t set the requires_grad attribute of the parameters to False.