transformers for language understanding. In NAACL-HLT, 2019.

[7] Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V Le. XLNet:

Generalized autoregressive pretraining for language understanding. In NeurIPS, 2019.

[8] Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. Albert:
A lite bert for self-supervised learning of language representations. In International Conference on Learning
Representations, 2020. URL https://openreview.net/forum?id=H1eA7AEtvS.

[9] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke
Zettlemoyer, and Veselin Stoyanov. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint
arXiv:1907.11692, 2019.

[10] Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. ELECTRA: Pre-training text

encoders as discriminators rather than generators. In ICLR, 2020.
LaMDA- Language Models for Dialog Applications