using IFT+EFT vs. IFT alone, e.g. the pairwise accuracy
agreement with humans increases from 65.1% to 78.7%.
Reward Modeling ability improves with Self-Training
We find that performing
a round of self-reward training improves the ability of the model at providing self-rewards
for the next iteration, in addition to its improved instruction following ability. Model M2
(Iteration 2) is trained using the reward model from M1 (Iteration 1), but provides improved
performance on all five metrics compared to M1. Fo