Do they introduce errors in the data or does the data already contain them?
Data already contain errors
Both grammatical error correction and text style transfer can be viewed as monolingual sequence-to-sequence transformation tasks, but the scarcity of directly annotated data for either task makes them unfeasible for most languages. We present an approach that does both tasks within the same trained model, and only uses regular language parallel data, without requiring error-corrected or style-adapted texts. We apply our model to three languages and present a thorough evaluation on both tasks, showing that the model is reliable for a number of error types and style transfer aspects.
Sequence-to-sequence (seq2seq) transformations have recently proven to be a successful framework for several natural language processing tasks, like: machine translation (MT) BIBREF0 , BIBREF1 , speech recognition BIBREF2 , speech synthesis BIBREF3 , natural language inference BIBREF4 and others. However, the success of these models depends on the availability of large amounts of directly annotated data for the task at hand (like translation examples, text segments and their speech recordings, etc.). This is a severe limitation for tasks where data is not abundantly available as well as for low-resource languages. Here we focus on two such tasks: grammatical error correction (GEC) and style transfer. Modern approaches to GEC learn from parallel corpora of erroneous segments and their manual corrections BIBREF5 , BIBREF6 ; text style transfer also relies on supervised approaches that require texts of the same meaning and different styles BIBREF7 , BIBREF8 or imprecise unsupervised methods BIBREF9 , BIBREF10 . In this paper we introduce an approach to performing both GEC and style transfer with the same trained model, while not using any supervised training data for either task. It is based on zero-shot neural machine translation (NMT) BIBREF11 , and as such, the only kind of data it uses is regular parallel corpora (with texts and their translations). However, we apply the model to do monolingual transfer, asking to translate the input segment into the same language. We show, that this “monolingual translation” is what enables the model to correct the errors in the input as well as adapt the output into a desired style. Moreover, the same trained model performs both tasks on several languages. Our main contributions are thus: (i) a single method for both style transfer and grammatical error correction, without using annotated data for either task, (ii) support for both tasks on multiple languages within the same model, (iii) a thorough quantitative and qualitative manual evaluation of the model on both tasks, and (iv) highlighting of the model's reliability aspects on both tasks. We used publicly available software and data; an online demo of our results is available, but concealed for anonymization purposes. We describe the details of our approach in Section SECREF2 , then evaluate it in terms of performance in grammatical error correction in Section SECREF3 and in style transfer in Section SECREF4 . The paper ends with a review of related work in Section SECREF5 and conclusions in Section SECREF6 .