Hi I am trying to set learning rate layerwise dynamically but getting below error.
Code:-
net.layers.append(torch.nn.Linear(len(net.layers)+input_neurons, 1))
if count == 1:
lr_policy.append({‘params’: net.layers[len(net.layers)-1].parameters(), ‘lr’: 0.001})
elif count > 2:
lr_policy.append({‘params’: net.layers[len(net.layers)-2].parameters(), ‘lr’: 0.2})
lr_policy.append({‘params’: net.layers[len(net.layers)-1].weight, ‘lr’: 0.005})
Logs:-
/anaconda3/lib/python3.7/site-packages/torch/optim/optimizer.py in add_param_group(self, param_group)
212
213         if not param_set.isdisjoint(set(param_group[‘params’])):
–> 214             raise ValueError(“some parameters appear in more than one parameter group”)
215
216         self.param_groups.append(param_group)
ValueError: some parameters appear in more than one parameter group