Separate optimizers will also allow you to have separate schedulers, in case e.g. you are fine-tuning a pretrained model, and you want to slowly ramp-up the learning rate for the low-level (first) part of the model, while the final part will have a higher learning rate from the beginning.