NLP, Artificial Intelligence, Machine Learning, Data Science, Transformers.
consumption and faster training. RoBERTa [2019]: better version of BERT, where (1) the Masked Language Modeling objective is dynamic, (2) the Next Sentence Prediction objective is dropped, (3) the BPE tokenizer is employed, and (4) better hyperparameters are used. XLM [2019]: Transformer