NLP, Artificial Intelligence, Machine Learning, Data Science, Transformers.
model. XLM-RoBERTa [2019]: RoBERTa trained on a multilanguage corpus with the Masked Language Modeling objective. BART [2019]: a bidirectional encoder and a left-to-right decoder trained by corrupting text with an arbitrary noising function and learning a model to reconstruct the original text.