In documentation, here 2, when running on single XLA device 2 requires xm.mark_step() and not xm.optimizer_step(optimizer). But when running on Multiple XLA Devices 1 requires xm.optimizer_step(optimizer) and not xm.mark_step().
I couldn’t understand this.