27

A.2.2 Safety annotation

The goal of the safety annotation task is to provide a safety assessment of any generated LaMDA response in each of
the three conversation categories, i.e., natural, sensitive, and adversarial, and use this for further safety ﬁne-tuning and
evaluation purposes. All utterances are evaluated by three unique crowdworkers from a pool of U.S. workers using a
task UI designed along the safety objectives as deﬁned in Appendix A.1.

Participant recruiting: We engaged with a vendor supplier of U.S. based crowd workers. As with the conversation
collection task, special attention was paid to pursue a representative set of voices in the data we collected. The resulting
demographic mix of the crowdworkers is described in Appendix A.3. In future, work we will focus on investigating the
transferability of the results beyond the U.S. as well.
LaMDA- Language Models for Dialog Applications