In my network, there are two models A and B. It runs like:
a = A(input)
b = B(a)

I want to freeze model A, only train model B. I want to know, are these enough to fix model A which has batchnorm layers and dropout layers:
for param in A.parameters():
    param.requires_grad = False
optimizer = torch.optim.Adam(B.parameters()……)

During my test, I found this code is also needed:
A.eval()

When I tell it to my friends, we both are confused. We think parameters in batchnorm layers will not be changed if we don’t optimize them (that means use an optimizer).
Batchnorm layers perform differently in eval and train mode. How about their parameters? Do the parameters change along with each prediction?