Alyssa Lees, Vinh Q Tran, Yi Tay, Jeﬀrey Sorensen, Jai Gupta, Donald Metzler, and Lucy Vasserman. A
new generation of perspective api: Eﬃcient multilingual character-level transformers. arXiv preprint
arXiv:2202.11176, 2022.

Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim
Krikun, Noam Shazeer, and Zhifeng Chen. Gshard: Scaling giant models with conditional computation
and automatic sharding. arXiv preprint arXiv:2006.16668, 2020.

Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-eﬃcient prompt tuning.

arXiv preprint arXiv:2104.08691, 2021.

Haoran Li, Abhinav Arora, Shuohui Chen, Anchit Gupta, Sonal Gupta, and Yashar Mehdad. Mtop: A
comprehensive multilingual task-oriented semantic parsing benchmark. arXiv preprint arXiv:2008.09335,
2020.
UL2- Unifying Language Learning Paradigms