25

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and
Kristina Toutanova. 2019. BERT: Pre-training of
deep bidirectional transformers for language under-
standing. In Proceedings of the 2019 Conference of
the North American Chapter of the Association for
Computational Linguistics: Human Language Tech-
nologies, Volume 1 (Long and Short Papers), pages
4171–4186, Minneapolis, Minnesota. Association for
Computational Linguistics.

Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiy-
ong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and
Zhifang Sui. 2022. A survey for in-context learning.
arXiv preprint arXiv:2301.00234.

Allyson Ettinger. 2020. What BERT Is Not: Lessons
from a New Suite of Psycholinguistic Diagnostics for
Language Models. Transactions of the Association
for Computational Linguistics, 8:34–48.

Tianyu Gao, Adam Fisch, and Danqi Chen. 2020.
Making pre-trained language models better few-shot
learners. arXiv preprint arXiv:2012.15723.
AreEmergentAbilitiesinLarge Language Models just In-Context