e see large gains from M2
to M3 through training using the preference data AIFT(M2) provided by the reward model
from Iteration 2.
Self-Rewarding models perform well on AlpacaEval 2 leaderboard
We evaluate
our models on the AlpacaEval 2.0 leaderboard format, with results given in Table 1. We
observe the same findings as in the head-to-head evaluations, that training iterations yield
improved win rates over GPT4-Turbo, from 9.94% in Iteration 1, to 15.38% in Iteration 2,
to 20.44% in Iteration 3. Our Iterati