I believe the working solution would be to make it verbose=False and log learning rate from optimizer every desirable logging_step.
Something like this:
logging_step = 1000
for step, batch in enumerate(dataloader):
    # some working logic here
    ...
    if step % logging_step == 0:
        print(f"{group['lr'] for group in optimizer.param_groups}")