Judge data. This is done by placing it in
the input format given in Figure 2, which consists of the scoring criteria description, and
the given instruction and response to be evaluated.1 For training targets, chain-of-thought
justifications and final scores out of 5 are not directly provided, so we use the SFT baseline
to generate such output evaluations for each input, and accept them into the training set if
the ranking of their scores agrees with the human rankings in the dataset. This results in
1775 tr