se it means
the increased capability of a model to self-reward does not affect its other skills. We can
thus use IFT+EFT training as Iteration 1 (M1) of our Self-Rewarding model, and then run
further iterations.
Iteration 2 (M2) improves over Iteration 1 (M1) and SFT Baseline
Iteration 2
of Self-Rewarding training (M2) provides superior instruction following to Iteration 1 (M1)
with 55.5% wins for M2 compared to only 11.7% for M1 in a head to head evaluation. It
provides similar gains over the SFT Baseline 