Self-supervised neural machine translation (SS-NMT) learns how to extract/select suitable training data from comparable -rather than parallel-corpora and how to translate, in a way that the two tasks support each other in a virtuous circle. SS-NMT has been shown to be competitive with state-of-the-art unsupervised NMT. In this study we provide an in-depth analysis of the sampling choices the SS-NMT model takes during training. We show that, without it having been told to do so, the model selects samples of increasing (i) complexity and (ii) task-relevance in combination with (iii) a denoising curriculum. We observe that the dynamics of the mutual-supervision of both system internal representation types is vital for the extraction and hence translation performance. We show that in terms of the human Gunning-Fog Readability index (GF), SS-NMT starts by extracting and learning from Wikipedia data suitable for high school (GF=10-11) and quickly moves towards content suitable for first year undergraduate students (GF=13).
This paper studies how to extract/select suitable training data from comparable —rather than parallel— corpora. The idea sounds reasonable.
My major concern is about the evaluation: it didn't compare with any existing work. Actually there quite a few papers  on mining parallel sentences from comparable corpora such as Wikipedia, as shown below. Seems the authors are not aware of those works and didn't review and compare with them. Without such comparisons, it is difficult to judge the effectiveness of the proposed method and the quality of this work. 
[1] Finding similar sentences across multiple languages in Wikipedia, Proceedings of the Workshop on NEW TEXT Wikis and blogs and other dynamic text sources. 2006.
[2] Method for building sentence-aligned corpus from wikipedia, 2008 AAAI Workshop on Wikipedia and Artificial Intelligence (WikiAI08). 2008.
[3] Extracting parallel sentences from comparable corpora using document-level alignment, Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 2010.
[4] "Improving machine translation performance by exploiting non-parallel corpora." Computational Linguistics2006.
[5] https://www.aclweb.org/anthology/W04-3208.pdf
[6] https://openreview.net/pdf?id=ryza73R9tQ
Minor issues:
	1. "For each language pair, a shared byte-pair encoding (BPE) (Sennrich et al., 2016) of 100k merge operations is applied." Most papers on neural machine translation don't use such a large BPE size, which is likely to lead to better performance. It would be better to use the same setting as previous work for fair comparisons.
	2. "In the case of SS-NMT, both tasks —data extraction and learning NMT— enable and enhance each other, such that this mutual supervision leads to a self-induced curriculum, which is the subject to our analysis." Similar idea, mutual boosting between data selection and model training, has been explored in the following paper, although not for machine translation. What's the difference between these two papers?
Learning to Teach, ICLR 2018.