3. Max Du
7. Victor Kolev
11. Mike Hardy
15. Chenchen Gu
19. Kaien Yang
23. Siyan Li

4. Kaili Huang
8. Karel D’Oosterlinck
12. Niveditha Iyer
16. Moritz Stephan
20. Ryan Chi
24. Amelia Hardy

7One volunteer did not respond for the DPO-PPO comparison.

27
Direct Preference Optimization