12

[25] D. Narayanan, M. Shoeybi, J. Casper, P. LeGresley, M. Patwary, V. Korthikanti, D. Vainbrand,
P. Kashinkunti, J. Bernauer, B. Catanzaro, A. Phanishayee, and M. Zaharia. Efficient large-scale
language model training on gpu clusters using megatron-lm. In Proceedings of the International
Conference for High Performance Computing, Networking, Storage and Analysis, SC ’21, New
York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450384421. doi:
10.1145/3458817.3476209. URL https://doi.org/10.1145/3458817.3476209.
Direct Preference Optimization