ll
yielded a head to head with the SFT Baseline of 29% wins vs 30% wins, i.e., no improvement.
6
Self-Rewarding M_3
vs.
SFT Baseline
Self-Rewarding M_2
vs.
SFT Baseline
Self-Rewarding M_1
vs.
SFT Baseline
62.5
49.2
30.5
27.7
36.3
38.7
9.8
14.5
30.9
Self-Rewarding Wins
Tie
SFT Baseline Wins
Self-Rewarding M_3
vs.
M_2
Self-Rewarding M_2
vs.
M_1
Self-Rewarding M_3
vs.
M_1
47.7
55.5
68.8
39.8
32.8
22.7
12.5
11.7
8.6
Left Wins (in Left vs. Right)
Tie
Right Wins
Figure 3: Instruction following ability improves wi