I have no idea either. Before torch 1.17, I noticed warmup was significantly longer and needed only 1 warmup step, but after 1.17, warmup is faster but needs 2 warmup steps. I know they wanted to make warmup faster in 1.17 but I’m not sure exactly why two steps are needed.