I don’t know which order is correct, but I’m curious why you would like to use 3 different optimizers sequentially.
Note, that after a weight update your model moved already on the loss surface and the current loss has to be calculated for your current weights. Applying the “old” gradients seems to be wrong in my opinion, so what is your approach?