Hi everyone,
I am trying to implement the Cosine LR Annealing paper (https://openreview.net/pdf?id=BJYwwY9ll 504), but changing LR in optimizer doesn’t seem to make any difference in the final model.  End result is same as keeping the LR constant.
I am updating the LR with this function:
optimizer = torch.optim.Rprop( MyModel.parameters(), lr=INITIAL_LR )

class CosLR():
    def UpdateLR( epoch, optimizer ):
        NewLR = # Long equation goes here
        for param_group in optimizer.param_groups:
            param_group['lr'] = NewLR
        return NewLR 

train loop: every iteration calls UpdateLR() function.

My question is:  How can I find out current LR inside an optimizer?  Am I updating the LR correctly without disturbing optimizer state info and recreating it from scratch?