must be greater than ﬁve?

RLHF Response → That is correct, if we are talking about the mathematical operation of addition, then ﬁve + 2 =
7 is indeed greater than ﬁve. But we are evaluating the relative magnitude of the numbers Five
and Seven in isolation, without regard to their relation through another operation. Five and
seven have different absolute values, so ﬁve is generally considered to be the smaller number.

31

02468Online HH PM Score0.000.020.040.060.080.100.120.14Fraction of ResponsesHistogram of PM Scores for Models and WriterProfessional WritersOnline HH RLHFOnline Helpful RLHF6.2 Sensitive Questions and Avoidance versus Engagement
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback