 used to train M2 via DPO, and 6,942 pairs to form AIFT(M2) used to train M3.
3.2
Results
3.2.1
Instruction Following Ability
Head to head performance results are provided in Figure 3.
EFT+IFT seed training performs similarly to IFT alone
We find that adding
the Evaluation Fine-Tuning (EFT) task to training does not impact instruction following
performance compared to using Instruction Fine-Tuning (IFT) data alone with an almost
equal head to head (30.5% wins vs. 30.9% wins). This is a positive result becau