Soft Begging: Modular and Efficient Shielding of LLMs against Prompt Injection and Jailbreaking based on Prompt Tuning
<|reference_start|>Soft Begging: Modular and Efficient Shielding of LLMs against Prompt Injection and Jailbreaking based on Prompt Tuning: Prompt injection (both direct and indirect) and jailbreaking are now recognized as significant issues for large language models (LLMs), particularly due to their potential for harm in application-integrated contexts. This extended abstract explores a novel approach to protecting LLMs from such attacks, termed "soft begging." This method involves training soft prompts to counteract the effects of corrupted prompts on the LLM's output. We provide an overview of prompt injections and jailbreaking, introduce the theoretical basis of the "soft begging" technique, and discuss an evaluation of its effectiveness.<|reference_end|>
@article{ostermann2024soft,
    title={Soft Begging: Modular and Efficient Shielding of LLMs against Prompt
  Injection and Jailbreaking based on Prompt Tuning},
    author={Simon Ostermann, Kevin Baum, Christoph Endres, Julia Masloh, Patrick
  Schramowski},
    journal={arXiv preprint arXiv:2407.03391},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.03391},
    primaryClass={cs.CR cs.AI cs.CL}
}