Hi, I think you may miss this description of pytorch_warmup 196 in the docs:
The warmup factor depends on Adam's `beta2` parameter for `RAdamWarmup` . Please see the original paper for the details.

The author says that the implementation depends on the parameter of Adam. So I think pytorch_warmup 196 cannot be used for SGD optimizer. Maybe I will check the code for more details when I am free.