I’d like to know how should I initialize the model if the model is separated into several modules.
For example:
model = def_model()  # backbone layers
model_loss = def_loss()  # FC classifier 
params =  list(model.parameters()) + list(model_loss.parameters())  # all the parameters
optimizer = torch.optim.SGD(params, lr)

Then if I want to train the model using apex fp16, which operation is correct?

Init all the sub-modules

[model, model_loss], optimizer = amp.initialize([model, model_loss], optimizer, opt_level="O1")


Only init the main module

model, optimizer = amp.initialize(model, optimizer, opt_level="O1")

Since the params already contains all the parameter, it is able to train using both methods. But what will happen if I only include partial modules in the init operation?