What are the restrictions of the restricted track?
The organizers provided a dataset allowed to use for training
We describe two entries from the Cambridge University Engineering Department to the BEA 2019 Shared Task on grammatical error correction. Our submission to the low-resource track is based on prior work on using finite state transducers together with strong neural language models. Our system for the restricted track is a purely neural system consisting of neural language models and neural machine translation models trained with back-translation and a combination of checkpoint averaging and fine-tuning -- without the help of any additional tools like spell checkers. The latter system has been used inside a separate system combination entry in cooperation with the Cambridge University Computer Lab.
The automatic correction of errors in text [In a such situaction INLINEFORM0 In such a situation] is receiving more and more attention from the natural language processing community. A series of competitions has been devoted to grammatical error correction (GEC): the CoNLL-2013 shared task BIBREF0 , the CoNLL-2014 shared task BIBREF1 , and finally the BEA 2019 shared task BIBREF2 . This paper presents the contributions from the Cambridge University Engineering Department to the latest GEC competition at the BEA 2019 workshop. We submitted systems to two different tracks. The low-resource track did not permit the use of parallel training data except a small development set with around 4K sentence pairs. For our low-resource system we extended our prior work on finite state transducer based GEC BIBREF3 to handle new error types such as punctuation errors as well as insertions and deletions of a small number of frequent words. For the restricted track, the organizers provided 1.2M pairs (560K without identity mappings) of corrected and uncorrected sentences. Our goal on the restricted track was to explore the potential of purely neural models for grammatical error correction. We confirm the results of BIBREF4 and report substantial gains by applying back-translation BIBREF5 to GEC – a data augmentation technique common in machine translation. Furthermore, we noticed that large parts of the training data do not match the target domain. We mitigated the domain gap by over-sampling the in-domain training corpus, and by fine-tuning through continued training. Our final model is an ensemble of four neural machine translation (NMT) models and two neural language models (LMs) with Transformer architecture BIBREF6 . Our purely neural system was also part of the joint submission with the Cambridge University Computer Lab described by BIBREF7 .