Doing multiple backward passes before calling any optimizers will give you end to end training too. It’s just a name for joint optimization. I didn’t have time to look into the paper, but it’s possible that you could backprop from Eq9 loss first, zero grads of Classifier, and then backprop from Eq8 + Eq10 (remember that .grad is accumulated instead of overwritten).