2.4 Comparing Models with Elo Scores

A signiﬁcant part of our analysis compares models against each other in order to generate associated Elo
scores, as described in [Askell et al., 2021]. That is, we have crowdworkers chat with two models simultane-
ously, with each model generating one response (either ‘A’ or ‘B’) at each turn, and we record the sample that
is preferred by the worker. This provides us with a record of ‘win rates’ between pairs of models, which we
can then ﬁt to corresponding Elo scores, to produce Figure 1 (where we show both win rates and Elo scores).
Two useful conversion formulas are

Win Fraction =

1
∆(Elo Score)

400

1 + 10

and ∆(Elo Score) ≈ 174 ∗ ∆(PM Score)

(2.1)
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback