You could use an unreduced loss (via passing reduction='none' to the criterion) and sort the loss values (and store the sort indices). Once they are sorted you could slice them, reduce the loss, call backward` on it, and update the parameters.
In the next step you could reuse the sort indices from the previous run and repeat the procedure using your schedule.