Different learning rate for a specific layer 
  
  
    Hello, it gets error when I resume the optimizer. 
I define the optimizer like that: 
     # 削减公用层learning_rate, 这个很重要
     all_parameters = set(model.parameters())
     nas_layers_params = []
     for m in model.modules():
         if isinstance(m, BlockSwitch):
             nas_layers_params += list(m.parameters())
     nas_layers_params = set(nas_layers_params)
     comm_layers_params = all_parameters - nas_layers_params
     nas_layers_params = list(nas_layers_params)
     comm_layers_params…