BertAdam's get_lr() not return correct learning rate
when the initial values of `BertAdam`'s `params` have `requires_grad=False` Parameter, it will just continue the loop in `step()` function line 251, after step() function, when I want to use `get_lr()` to get the current learning rate, the state of this Parameter is a empty dict, so the function just return `[0]` in line 230. I think it is out of expectation, maybe more checking of the grad is needed somewhere.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
