ion A.1. We find good
overlap between the IFT and AIFT(M1) examples, which is desired, while the EFT examples
lie in a different part of the embedding space. We observe that generations from M1 have
an average length of 1092, for M2 they are 1552, and for M3 they are 2552, so the model is
learning to generate longer responses, which we note may be a factor in relative performance.
3.2.2
Reward Modeling Ability
Reward modeling evaluation results are provided in Table 2.
EFT augmentation improves over SFT bas