title: GuardReasoner: Towards Reasoning-based LLM Safeguards
summary: As LLMs increasingly impact safety-critical applications, ensuring their
  safety using guardrails remains a key challenge. This paper proposes
  GuardReasoner, a new safeguard for LLMs, by guiding the guard model to learn to
  reason. Concretely, we first create the GuardReasonerTrain dataset, which
  consists of 127K samples with 460K detailed reasoning steps. Then, we introduce
  reasoning SFT to unlock the reasoning capability of guard models. In addition,
  we present hard sample DPO to further strengthen their reasoning ability. In
  this manner, GuardReasoner achieves better performance, explainability, and
  generalizability. Extensive experiments and analyses on 13 benchmarks of 3
  guardrail tasks demonstrate its superiority. Remarkably, GuardReasoner 8B
  surpasses GPT-4o+CoT by 5.74% and LLaMA Guard 3 8B by 20.84% F1 score on
  average. We release the training data, code, and models with different scales
  (1B, 3B, 8B) of GuardReasoner : https://github.com/yueliu1999/GuardReasoner/.
paper_authors._id: 679c4ac5e2c0dbf282597d35
paper_authors.name: Yue Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-03T12:37:21.603Z
paper_authors._id: 6650c77a74664a42ddfb9187
paper_authors.avatarUrl: /avatars/92001bbe0ae9b14309730316b639cede.svg
paper_authors.fullname: yueliu1999
paper_authors.type: user
paper_authors.user: yueliu1999
paper_authors._id: 679c4ac5e2c0dbf282597d36
paper_authors.name: Hongcheng Gao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-31T08:35:51.645Z
paper_authors._id: 62728f4f6253fe2068da1021
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62728f4f6253fe2068da1021/KZ65X0EH98AF3zXemPiap.jpeg
paper_authors.fullname: Hongcheng Gao
paper_authors.type: user
paper_authors.user: HongchengGao
paper_authors._id: 679c4ac5e2c0dbf282597d37
paper_authors.name: Shengfang Zhai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-31T08:41:32.474Z
paper_authors._id: 6366429195204b4649c658b8
paper_authors.avatarUrl: /avatars/5d80e9ebe0b57fd815f36796b9187248.svg
paper_authors.fullname: Shengfang Zhai
paper_authors.type: user
paper_authors.user: zsf
paper_authors._id: 679c4ac5e2c0dbf282597d38
paper_authors.name: Jun Xia
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-31T08:41:53.366Z
paper_authors._id: 679c68bbfc30f43de85206f5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/IJWda9ZYtjzlhr2ehsLHu.jpeg
paper_authors.fullname: Jun Xia
paper_authors.type: user
paper_authors.user: JunXia97
paper_authors._id: 679c4ac5e2c0dbf282597d39
paper_authors.name: Tianyi Wu
paper_authors._id: 679c4ac5e2c0dbf282597d3a
paper_authors.name: Zhiwei Xue
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-31T08:42:30.842Z
paper_authors._id: 63f42ca3520c1461892ee929
paper_authors.avatarUrl: /avatars/095241acfe7c783d2406abf63ff81f65.svg
paper_authors.fullname: xuezhiwei
paper_authors.type: user
paper_authors.user: lakxtxue
paper_authors._id: 679c4ac5e2c0dbf282597d3b
paper_authors.name: Yulin Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-31T08:42:41.013Z
paper_authors._id: 65efc25828426de60f977dfc
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/u8ZcIoo58JPLdnjm-jZeo.png
paper_authors.fullname: Yulin Chen
paper_authors.type: user
paper_authors.user: CallMeChen
paper_authors._id: 679c4ac5e2c0dbf282597d3c
paper_authors.name: Kenji Kawaguchi
paper_authors._id: 679c4ac5e2c0dbf282597d3d
paper_authors.name: Jiaheng Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-31T08:37:04.493Z
paper_authors._id: 669e19e5dac1eb34c0f5f505
paper_authors.avatarUrl: /avatars/bec7d1d1dac2ad6570844d1f00e7df0a.svg
paper_authors.fullname: Jiaheng Zhang
paper_authors.type: user
paper_authors.user: jiaheng233
paper_authors._id: 679c4ac5e2c0dbf282597d3e
paper_authors.name: Bryan Hooi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-31T08:42:50.273Z
paper_authors._id: 651d8032c50012d33e914f2f
paper_authors.avatarUrl: /avatars/0a44c9f51fc50ce86582e328c361ea00.svg
paper_authors.fullname: Bryan Hooi
paper_authors.type: user
paper_authors.user: bhooi
paper_title: GuardReasoner: Towards Reasoning-based LLM Safeguards
paper_summary: As LLMs increasingly impact safety-critical applications, ensuring their
  safety using guardrails remains a key challenge. This paper proposes
  GuardReasoner, a new safeguard for LLMs, by guiding the guard model to learn to
  reason. Concretely, we first create the GuardReasonerTrain dataset, which
  consists of 127K samples with 460K detailed reasoning steps. Then, we introduce
  reasoning SFT to unlock the reasoning capability of guard models. In addition,
  we present hard sample DPO to further strengthen their reasoning ability. In
  this manner, GuardReasoner achieves better performance, explainability, and
  generalizability. Extensive experiments and analyses on 13 benchmarks of 3
  guardrail tasks demonstrate its superiority. Remarkably, GuardReasoner 8B
  surpasses GPT-4o+CoT by 5.74% and LLaMA Guard 3 8B by 20.84% F1 score on
  average. We release the training data, code, and models with different scales
  (1B, 3B, 8B) of GuardReasoner : https://github.com/yueliu1999/GuardReasoner/.
paper_ai_summary: GuardReasoner enhances LLM safety through enhanced reasoning capabilities using specialized training datasets and techniques, outperforming existing models on guardrail tasks.
paper_ai_keywords: LLMs guardrails GuardReasoner GuardReasonerTrain dataset reasoning SFT hard sample DPO explainability generalizability F1 score GPT-4o+CoT LLaMA Guard 3 8B