I finally read the paper. From what I understand, they simply train two Seq2Seq models side by side: one processing sentences left to right, the other from right to left.
As far as I can tell, the training of both models is done in parallel, just with opposite sequence orders. For inference, the run an incorrect input sentence alternatively through both models until the output is “good enough” or doesn’t change.
In short, you just have to train two models – well, one model in terms of its architecture but with different datasets, where the difference is solely in the order of the sentences.