model sizes, ranging from 13M to 52B parameters. Compared to the baseline model (a pre-trained LM
ﬁnetuned on Python code), the RLHF model is more capable of distinguishing quality (right), but is worse
at language modeling (left).
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback