Thank you! What you said is reasonable and matches what’s implemented in lr_finder fast_ai package.They basically train a number of batches equal to the number of gradient accumulation steps. Then they do scheduler.step().