title: Chain-of-Defensive-Thought: Structured Reasoning Elicits Robustness in
    Large Language Models against Reference Corruption
summary: Chain-of-thought prompting has demonstrated great success in facilitating the
  reasoning abilities of large language models. In this work, we explore how
  these enhanced reasoning abilities can be exploited to improve the robustness
  of large language models in tasks that are not necessarily reasoning-focused.
  In particular, we show how a wide range of large language models exhibit
  significantly improved robustness against reference corruption using a simple
  method called chain-of-defensive-thought, where only a few exemplars with
  structured and defensive reasoning are provided as demonstrations. Empirically,
  the improvements can be astounding, especially given the simplicity and
  applicability of the method. For example, in the Natural Questions task, the
  accuracy of GPT-4o degrades from 60% to as low as 3% with standard prompting
  when 1 out of 10 references provided is corrupted with prompt injection
  attacks. In contrast, GPT-4o using chain-of-defensive-thought prompting
  maintains an accuracy of 50%.
paper_authors._id: 68129287421d23bcbdbc535f
paper_authors.name: Wenxiao Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-02T06:35:58.788Z
paper_authors._id: 659dc02d72238596c24d49f5
paper_authors.avatarUrl: /avatars/d4600d23ccc72f296fab7f626d5895e7.svg
paper_authors.fullname: Wenxiao Wang
paper_authors.type: user
paper_authors.user: wangwenxiao
paper_authors._id: 68129287421d23bcbdbc5360
paper_authors.name: Parsa Hosseini
paper_authors._id: 68129287421d23bcbdbc5361
paper_authors.name: Soheil Feizi
paper_title: Chain-of-Defensive-Thought: Structured Reasoning Elicits Robustness in
    Large Language Models against Reference Corruption
paper_summary: Chain-of-thought prompting has demonstrated great success in facilitating the
  reasoning abilities of large language models. In this work, we explore how
  these enhanced reasoning abilities can be exploited to improve the robustness
  of large language models in tasks that are not necessarily reasoning-focused.
  In particular, we show how a wide range of large language models exhibit
  significantly improved robustness against reference corruption using a simple
  method called chain-of-defensive-thought, where only a few exemplars with
  structured and defensive reasoning are provided as demonstrations. Empirically,
  the improvements can be astounding, especially given the simplicity and
  applicability of the method. For example, in the Natural Questions task, the
  accuracy of GPT-4o degrades from 60% to as low as 3% with standard prompting
  when 1 out of 10 references provided is corrupted with prompt injection
  attacks. In contrast, GPT-4o using chain-of-defensive-thought prompting
  maintains an accuracy of 50%.
paper_ai_summary: Chain-of-defensive-thought prompting enhances the robustness of large language models against reference corruption in non-reasoning tasks by using simple defensive reasoning exemplars.
paper_ai_keywords: chain-of-thought large language models robustness reference corruption chain-of-defensive-thought prompt injection attacks Natural Questions task GPT-4o