I read it the same way" at each iteration, a new model is initialized from scratch, then pseudo-trained (PT) on the artificial data, then fine-tuned (FT) on the real data
That is coherent with how they present it in Section 2:
In our preliminary experiments, we find that the separate training strategy with the whole pseudo parallel dataset (i.e. S = {(x, fθ(x))|x ∈ U}) produces better or equal performance for neural sequence generation while being simpler. Therefore, in the remainder of this paper we use this simpler setting.