63

were similar, MTurk workers tended to opt in to more work and their pay structure incentivized quicker
conversations.
We communicated with select workers on a daily basis over Slack. We used this channel to announce new
tasks and provide guidance; we discussed difﬁcult edge cases with the group as they came up. At an interme-
diate stage of the project we provided some additional thoughts on more advanced forms of interaction with
the model; the slack message we sent is shown in Figure 43. Our workers alerted us to bugs and performance
issues as they ran into them.
We sent both groups of workers a demographics survey, and the results are shown in ﬁgure 44. Survey re-
sponses were anonymous, as we did not collect any personal identiﬁable information alongside demographic
information.

D.2

Instructions and Interface
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback