reas the other parts of the creation pipeline (generating the
response, and evaluating it) use the model being trained. For candidate response generation
we sample N = 4 candidate responses with temperature T = 0.7, p = 0.9. When evaluating
candidate responses, as there is variance to these scores, in our experiments we also use
sampled decoding (with the same parameters) and generate these evaluations multiple (3)
times and take the average. We added 3,964 such preference pairs to form the AIFT(M1)
dataset