Chain-of-Defensive-Thought: Structured Reasoning Elicits Robustness in
  Large Language Models against Reference Corruption
Chain-of-thought prompting has demonstrated great success in facilitating the
reasoning abilities of large language models. In this work, we explore how
these enhanced reasoning abilities can be exploited to improve the robustness
of large language models in tasks that are not necessarily reasoning-focused.
In particular, we show how a wide range of large language models exhibit
significantly improved robustness against reference corruption using a simple
method called chain-of-defensive-thought, where only a few exemplars with
structured and defensive reasoning are provided as demonstrations. Empirically,
the improvements can be astounding, especially given the simplicity and
applicability of the method. For example, in the Natural Questions task, the
accuracy of GPT-4o degrades from 60% to as low as 3% with standard prompting
when 1 out of 10 references provided is corrupted with prompt injection
attacks. In contrast, GPT-4o using chain-of-defensive-thought prompting
maintains an accuracy of 50%.
68129287421d23bcbdbc535f
Wenxiao Wang
claimed_verified
2025-05-02T06:35:58.788Z
659dc02d72238596c24d49f5
/avatars/d4600d23ccc72f296fab7f626d5895e7.svg
Wenxiao Wang
user
wangwenxiao
68129287421d23bcbdbc5360
Parsa Hosseini
68129287421d23bcbdbc5361
Soheil Feizi
Chain-of-Defensive-Thought: Structured Reasoning Elicits Robustness in
  Large Language Models against Reference Corruption
Chain-of-thought prompting has demonstrated great success in facilitating the
reasoning abilities of large language models. In this work, we explore how
these enhanced reasoning abilities can be exploited to improve the robustness
of large language models in tasks that are not necessarily reasoning-focused.
In particular, we show how a wide range of large language models exhibit
significantly improved robustness against reference corruption using a simple
method called chain-of-defensive-thought, where only a few exemplars with
structured and defensive reasoning are provided as demonstrations. Empirically,
the improvements can be astounding, especially given the simplicity and
applicability of the method. For example, in the Natural Questions task, the
accuracy of GPT-4o degrades from 60% to as low as 3% with standard prompting
when 1 out of 10 references provided is corrupted with prompt injection
attacks. In contrast, GPT-4o using chain-of-defensive-thought prompting
maintains an accuracy of 50%.
Chain-of-defensive-thought prompting enhances the robustness of large language models against reference corruption in non-reasoning tasks by using simple defensive reasoning exemplars.
chain-of-thought large language models robustness reference corruption chain-of-defensive-thought prompt injection attacks Natural Questions task GPT-4o