Learning rate schedules improvement + extension
re: [PR#389](https://github.com/huggingface/pytorch-pretrained-BERT/pull/389)
- refactored learning rate schedules into objects
- added `WarmupCosineWithHardRestartsSchedule` for cosine schedule with hard restarts
- added `WarmupCosineWithWarmupRestartsSchedule` for cosine schedule with restarts where each restart uses the same warmup slope
Sorry for the delay in reviewing this.
This is a great PR and it looks good to me.
Thanks for adding some tests also.
I agree with the (mostly cosmetic) comments from @marpaia.
Do you think you can fix them and then we can merge?<|||||>Fixed @marpaia 's comments.<|||||>Awesome @lukovnikov, I think it looks great! Thanks for making this already awesome library even more awesome 💯 <|||||>Thanks @lukovnikov!