On top of my head, there are two options:

write your own lr scheduler (see examples here:https://github.com/pytorch/pytorch/blob/master/torch/optim/lr_scheduler.py 468)
use different optimizers for different parts of your network.