Poison

2

Benign

Poison

effectiveness of our approach. In summary, all above results
demonstrate the risks of the unauthorized third-party NLP
generative models.

IV. CONCLUSION AND FUTURE PLANS

In this paper, we present the first backdoor attack on RL
fine-tuning in language models and propose a new attack
method called BadGPT. Our experiments on a benchmark
model show that this new NLP paradigm introduces security
vulnerabilities. We aim to raise awareness of these risks and
plan to extend our work by evaluating BadGPT on larger-scale
models, exploring more advanced attacks for real scenarios,
and developing effective defenses against backdoor attacks
on RL fine-tuning in language models. This research has
significant implications for the security of NLP systems and
highlights the need for further research in this area.

REFERENCES

[1] Schulman, J., et al. “ChatGPT: Optimizing language models for dialogue.”

(2022).
BadGPT- Exploring Security Vulnerabilities of ChatGPT via Backdoor Attacks to InstructGPT