pre-trained language models via knowledge distillation. arXiv preprint arXiv:2109.06243 (2021).

[257] Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. CommonsenseQA: A Question Answering Challenge Targeting Commonsense

Knowledge. In ACL. ACL, Minneapolis, Minnesota, 4149–4158.

[258] Min Tang, Xiaoqiang Luo, and Salim Roukos. 2002. Active learning for statistical natural language parsing. In ACL. 120–127.
[259] Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford Alpaca:

An Instruction-following LLaMA model. https://github.com/tatsu-lab/stanford_alpaca.

[260] Yi Tay, Mostafa Dehghani, Samira Abnar, Hyung Won Chung, William Fedus, Jinfeng Rao, Sharan Narang, Vinh Q Tran, Dani Yogatama, and Donald

Metzler. 2022. Scaling laws vs model architectures: How does inductive bias influence scaling? arXiv preprint arXiv:2207.10551 (2022).
TheEfficiencySpectrumofLargeLanguageModels-AnAlgorithmicSurvey