dousse, Amanda Askell, Anna Chen, Nova DasSarma,
Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful
and harmless assistant with reinforcement learning from human feedback. arXiv preprint
arXiv:2204.05862, 2022a.
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy
Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitu-
tional AI: Harmlessness from AI feedback. arXiv preprint arXiv:2212.08073, 2022b.
Yushi Bai, Jiahao Yi