the highest ranked by humans.
3.1.3
Training Details
Instruction following training
The training hyperparameters we use are as follows.
For SFT we use learning rate 5.5e−6 which linearly decays to 1.1e−6, batch size 16 and
dropout 0.1. We only calculate the loss on target tokens instead of the full sequence. For
DPO we use learning rate 1e−6 which linearly decays to 1e−7, batch size 16, dropout 0.1,
and a β value of 0.1. We perform early stopping by saving a checkpoint every 200 steps
and evaluating generat