It depends how you would like to reload and use these models afterwards.
E.g. if you only care about the two submodules and would like to use them afterwards, the first approach would work. Note that you would lose the self.classifier as it’s a module in MyEnsemble.
In the latter case, all parameters will be stored and you would have to recreate the ensemble model to load the state_dict again (you could also manipulate the state_dict manually, but I would consider this a workaround and not a proper way to restore the parameters).