Context Distillation with Answer Templates During the prompt collection phase, we also asked annotators
to label prompts according to risk categories, which enables even more targeted preprompts. Specifically,
this allows us to provide some dedicated answer templates of how adversarial prompts should be addressed,
based on each identified risk category. Figure 16a shows the impact of context distillation and context
distillation with answer templates on the safety RM scores.

27

(a) Impact on Safety RM Score.

(b) Targeted Context Distillation.
Llama2