Did you call  model.eval()  before performing the evaluation?

Yes.

Have you trained the freezer parts before?

Also yes!
But why would the optimizer change these parameters when I didn’t add them in the first place? I only added parameters with requires_grad == True to the optimizer.