You can set all learning rates by doing
for group in optim.param_groups:
    group["lr"] = 0.001

Also using a learning rate scheduler might be helpful as well.
I hope these solutions would be helpful.