r example, pairwise accuracy improves
from 78.7% to 80.4%. Iteration 3 (M3) improves several of these metrics further compared
7
Table 1: AlpacaEval 2.0 results (win rate over GPT-4 Turbo evaluated by GPT-4).
Self-Rewarding iterations yield improving win rates. Iteration 3 (M3) outperforms many
existing models that use proprietary training data or targets distilled from stronger models.
Alignment Targets
Model
Win Rate
Distilled
Proprietary
Self-Rewarding 70B
Iteration 1 (M1)
9.94%
Iteration 2 (M2)
15.38%
I