Hello,
The following topic (Two optimizers for one model 199) discussed what you can do with two optimizers. You only have to specify the correct parameters to each optimizers. For example:
optim1 = torch.optim.SGD(model.parameters(), lr=0.001)
optim2 = torch.optim.Adam(model.conv3.parameters(), lr=0.05)

Good luck!