Giving multiple parameters in optimizer
How can i give multiple parameters to the optimizer? <SCODE>fc1 = nn.Linear(784, 500)
fc2 = nn.Linear(500, 10)
optimizer = torch.optim.SGD([fc1.parameters(), fc2.parameters()], lr=0.01)  # This causes an error.
<ECODE> In this case, for simplicity, i don’t want to use a class with nn.Module.
yunjey
you have to concatenate python lists: <SCODE>params = list(fc1.parameters()) + list(fc2.parameters())

torch.optim.SGD(params, lr=0.01)
<ECODE>
smth
Thanks, it works well.
yunjey
Zichun_Zhang
Dear Soumith, While executing your approach, it says: TypeError: add() received an invalid combination of arguments - got (list), but expected one of: (Tensor other, Number alpha) (Number other, Number alpha) Can you help me?) Is there something wrong?
MrPositron
Probably you set a bracket to the wrong place. You have to convert the parameters to a list separately and add the lists afterwards.
justusschock
[SOLVED] <SCODE>params = self.net.state_dict()
pas = list(params['net.0.weight']) + list(params['net.0.bias']) + list(params['net.3.weight'] + list(params['net.3.bias'])  + list(params['net.6.weight']) + list(params['net.6.bias']))
self.optimizer1 = optim.Adam(pas, lr = 0.01)
<ECODE> Here is my code. I think everything is ok
MrPositron
<SCODE>import itertools

params = [fc1.parameters(), fc2.parameters()]

torch.optim.SGD(itertools.chain(*params), lr=0.01)
<ECODE>
huangbiubiu
How is this different from just putting all of the tensors in a list directly as OP did?
riley
If your models are in a list or tuple somewhere already, you can also use a nested list comprehension: <SCODE>models = [nn.Linear(784, 500),
          nn.Linear(500, 10)
          ]
optimizer = torch.optim.SGD((par for model in models for par in model.parameters()),
                             lr=0.01)
<ECODE>
NeverConvex
OP is passing a list of lists.
Shubajit_Saha
Thank you! This helped me a lot!
ncuxomun