Thank you for your response. In the MAML setting, we need two sets of parameters to be preserved, so I use one of them as model.parameters() and the other loaded with nn.functional operations.
Because eventually the loss comes from forward propagations through both parameters, the computational graphs need to be preserved as well, so it can’t be done with load_state_dict operations, or storing them in separate models.
This is related to the discussion here: Gradient computation in meta-learning algorithms 2