policy optimization algorithms. CoRR, abs/1707.06347.

[Solaiman and Dennison, 2021] Solaiman, I. and Dennison, C. (2021). Process for adapting language models

to society (PALMS) with values-targeted datasets. CoRR, abs/2106.10328.

73

[Stiennon et al., 2020] Stiennon, N., Ouyang, L., Wu, J., Ziegler, D. M., Lowe, R., Voss, C., Radford, A.,

Amodei, D., and Christiano, P. (2020). Learning to summarize from human feedback.
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback