Is there a way to use two optimizers for one model in more beautiful way?
Now, as I understand we should do something like these:
net = Model()
part_1_parameters = ...
part_2_parameters = ...
opt1 = optimizer_1(part_1_parameters)
opt2 = optimizer_2(part_1_parameters)

### train epoch
opt1.zero_grad()
opt2.zero_grad()

loss = ...
loss.backward()

opt1.step()
opt2.step()