(2021). On-the-ﬂy controlled text generation with experts and anti-experts. CoRR, abs/2105.03023.

[Liu et al., 2020] Liu, J. Z., Lin, Z., Padhy, S., Tran, D., Bedrax-Weiss, T., and Lakshminarayanan, B. (2020).

Simple and principled uncertainty estimation with deterministic deep learning via distance awareness.

[Menick et al., 2022] Menick, J., Trebacz, M., Mikulik, V., Aslanides, J., Song, F., Chadwick, M., Glaese,
M., Young, S., Campbell-Gillingham, L., Irving, G., and McAleese, N. (2022). Teaching language models
to support answers with veriﬁed quotes.

[Mihaylov et al., 2018] Mihaylov, T., Clark, P., Khot, T., and Sabharwal, A. (2018). Can a suit of armor

conduct electricity? a new dataset for open book question answering. In EMNLP.

72
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback