Exploring Straightforward Conversational Red-Teaming
<|reference_start|>Exploring Straightforward Conversational Red-Teaming: Large language models (LLMs) are increasingly used in business dialogue systems but they pose security and ethical risks. Multi-turn conversations, where context influences the model's behavior, can be exploited to produce undesired responses. In this paper, we examine the effectiveness of utilizing off-the-shelf LLMs in straightforward red-teaming approaches, where an attacker LLM aims to elicit undesired output from a target LLM, comparing both single-turn and conversational red-teaming tactics. Our experiments offer insights into various usage strategies that significantly affect their performance as red teamers. They suggest that off-the-shelf models can act as effective red teamers and even adjust their attack strategy based on past attempts, although their effectiveness decreases with greater alignment.<|reference_end|>
@article{kour2024exploring,
    title={Exploring Straightforward Conversational Red-Teaming},
    author={George Kour and Naama Zwerdling and Marcel Zalmanovici and Ateret
  Anaby-Tavor and Ora Nova Fandina and Eitan Farchi},
    journal={arXiv preprint arXiv:2409.04822},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.04822},
    primaryClass={cs.CL cs.AI}
}