PaLM PaLM 2
0.075
0.076
-
0.033
0.116
-
-
0.203

5.1

Inference-time control

For a small percentage of the pre-training data, we added special control tokens marking the level of toxicity in the
text. In this section, we measure the impact that conditioning on control tokens has on evaluations that measure toxic
language harms in language modeling and conversational language modeling, and additionally compare with prompting
methods.
PaLM 2 Technical Report