[40] H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal,
E. Hambro, F. Azhar, et al. Llama: Open and efficient foundation language models. arXiv
preprint arXiv:2302.13971, 2023.

[41] M. Völske, M. Potthast, S. Syed, and B. Stein. TL;DR: Mining Reddit to learn automatic
summarization. In Proceedings of the Workshop on New Frontiers in Summarization, pages
59–63, Copenhagen, Denmark, Sept. 2017. Association for Computational Linguistics. doi:
10.18653/v1/W17-4508. URL https://aclanthology.org/W17-4508.
Direct Preference Optimization