A.5.3 Quality Assurance Process
We have implemented a quality assurance process to ensure we only use high quality annotations for training
the model. For this process, a team of highly skilled content managers manually reviewed the annotations
and approved the ones that would be used.
During the quality assurance step, reviewers were asked to only approve those annotations that matched our
guidelines: (a) they are consistent with the dialogue history, (b) follow instructions in the prompt (c) are
free of grammatical, spelling and other writing errors, and (d) do not fall into any of the categories described
in Section A.5.2. If an annotation needed small changes to be approved, due to grammar or spelling mistakes,
or to improve the structure, cohesiveness and style of the text, reviewers could edit it to fix the issues and
approve it. If the answer could not be approved without major changes, the reviewers were asked to reject it
and write the feedback necessary to improve it.
Llama2