NLP, Artificial Intelligence, Machine Learning, Data Science, Transformers.
pre-trained on a corpus of several languages using objectives like Causal Language Modeling, Masked Language Modeling, and Translation Language Modeling. XLNet [2019]: Transformer-XL with a generalized autoregressive pre-training method that enables learning bidirectional dependences. PEGASUS