The warmup technique is used during the first phase of the training. It consists in small learning rate and a scheduler that increases it everytime until a certain number of steps (warmup steps). So you need 2 different scheduler, as you can see from the code you posted, the warmup scheduler seems to be used only during the first epoch.
reference: https://datascience.stackexchange.com/questions/55991/in-the-context-of-deep-learning-what-is-training-warmup-steps 10