ets and rewards from the model itself for further iterations of training.
Preference optimization outperforms augmenting with positive examples only
We also tried the alternative self-training procedure of adding high-quality self-instruction
creation examples to supervised fine-tuning (without preference optimization). Unfortunately
we could not find a configuration where this approach helped. For example, adding 11,254
such examples that scored 5 out of 5, and optimizing the mixing weight in training, sti