Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasu-
pat, and Ming-Wei Chang. 2020. Realm: Retrieval-
augmented language model pre-training.

Junxian He, Jiatao Gu, Jiajun Shen, and Marc’Aurelio
Ranzato. 2020. Revisiting self-training for neural
In International Conference
sequence generation.
on Learning Representations.

Or Honovich, Thomas Scialom, Omer Levy, and Timo
Schick. 2022. Unnatural instructions: Tuning lan-
guage models with (almost) no human labor.

Gautier Izacard and Edouard Grave. 2021. Distilling
knowledge from reader to retriever for question an-
swering. In International Conference on Learning
Representations.

Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas
Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-
Yu, Armand Joulin, Sebastian Riedel, and Edouard
Grave. 2022. Atlas: Few-shot learning with retrieval
augmented language models.
Toolformer