Paul F Christiano, Jan Leike, Tom Brown, Miljan Mar-
tic, Shane Legg, and Dario Amodei. 2017. Deep
reinforcement learning from human preferences. In
Advances in Neural Information Processing Systems,
volume 30. Curran Associates, Inc.

Hyung Won Chung, Le Hou, S. Longpre, Barret Zoph,
Yi Tay, William Fedus, Eric Li, Xuezhi Wang,
Mostafa Dehghani, Siddhartha Brahma, Albert Web-
son, Shixiang Shane Gu, Zhuyun Dai, Mirac Suz-
gun, Xinyun Chen, Aakanksha Chowdhery, Dasha
Valter, Sharan Narang, Gaurav Mishra, Adams Wei
Yu, Vincent Zhao, Yanping Huang, Andrew M.
Dai, Hongkun Yu, Slav Petrov, Ed Huai hsin Chi,
Jeff Dean, Jacob Devlin, Adam Roberts, Denny
Zhou, Quoc V. Le, and Jason Wei. 2022. Scal-
ing instruction-ﬁnetuned language models. ArXiv,
abs/2210.11416.

Lavina Daryanani. 2023.

How to jailbreak chat-
https://watcher.guru/news/

gpt.
how-to-jailbreak-chatgpt.
Multi-step Jailbreaking Privacy Attacks on ChatGPT