I am looking for a way to do Epoch warm-ups/ learning rate warmups with SGD, but I can’t find anything useful. The best thing I could find was this site: https://github.com/Tony-Y/pytorch_warmup 334 but this is for ADAM and I am looking for a way to do this SGD. can someone point me in the right direction?