I’m not sure whether that is  what you want or not, but if you have modules with named parameters, you can do something like:
L = nn.Linear(5,3)
optim.SGD([
                {'params': L.weight },
                {'params': L.bias, 'lr': 1e-3}
            ], lr=1e-2, momentum=0.9)

You can read torch.optim doc 1 for more information.