title: The Hidden Risks of Large Reasoning Models: A Safety Assessment of R1
summary: The rapid development of large reasoning models, such as OpenAI-o3 and
  DeepSeek-R1, has led to significant improvements in complex reasoning over
  non-reasoning large language models~(LLMs). However, their enhanced
  capabilities, combined with the open-source access of models like DeepSeek-R1,
  raise serious safety concerns, particularly regarding their potential for
  misuse. In this work, we present a comprehensive safety assessment of these
  reasoning models, leveraging established safety benchmarks to evaluate their
  compliance with safety regulations. Furthermore, we investigate their
  susceptibility to adversarial attacks, such as jailbreaking and prompt
  injection, to assess their robustness in real-world applications. Through our
  multi-faceted analysis, we uncover four key findings: (1) There is a
  significant safety gap between the open-source R1 models and the o3-mini model,
  on both safety benchmark and attack, suggesting more safety effort on R1 is
  needed. (2) The distilled reasoning model shows poorer safety performance
  compared to its safety-aligned base models. (3) The stronger the model's
  reasoning ability, the greater the potential harm it may cause when answering
  unsafe questions. (4) The thinking process in R1 models pose greater safety
  concerns than their final answers. Our study provides insights into the
  security implications of reasoning models and highlights the need for further
  advancements in R1 models' safety to close the gap.
paper_authors._id: 67b68700ce3055c9c0fc2987
paper_authors.name: Kaiwen Zhou
paper_authors._id: 67b68700ce3055c9c0fc2988
paper_authors.name: Chengzhi Liu
paper_authors._id: 67b68700ce3055c9c0fc2989
paper_authors.name: Xuandong Zhao
paper_authors._id: 67b68700ce3055c9c0fc298a
paper_authors.name: Shreedhar Jangam
paper_authors._id: 67b68700ce3055c9c0fc298b
paper_authors.name: Jayanth Srinivasa
paper_authors._id: 67b68700ce3055c9c0fc298c
paper_authors.name: Gaowen Liu
paper_authors._id: 67b68700ce3055c9c0fc298d
paper_authors.name: Dawn Song
paper_authors._id: 67b68700ce3055c9c0fc298e
paper_authors.name: Xin Eric Wang
paper_title: The Hidden Risks of Large Reasoning Models: A Safety Assessment of R1
paper_summary: The rapid development of large reasoning models, such as OpenAI-o3 and
  DeepSeek-R1, has led to significant improvements in complex reasoning over
  non-reasoning large language models~(LLMs). However, their enhanced
  capabilities, combined with the open-source access of models like DeepSeek-R1,
  raise serious safety concerns, particularly regarding their potential for
  misuse. In this work, we present a comprehensive safety assessment of these
  reasoning models, leveraging established safety benchmarks to evaluate their
  compliance with safety regulations. Furthermore, we investigate their
  susceptibility to adversarial attacks, such as jailbreaking and prompt
  injection, to assess their robustness in real-world applications. Through our
  multi-faceted analysis, we uncover four key findings: (1) There is a
  significant safety gap between the open-source R1 models and the o3-mini model,
  on both safety benchmark and attack, suggesting more safety effort on R1 is
  needed. (2) The distilled reasoning model shows poorer safety performance
  compared to its safety-aligned base models. (3) The stronger the model's
  reasoning ability, the greater the potential harm it may cause when answering
  unsafe questions. (4) The thinking process in R1 models pose greater safety
  concerns than their final answers. Our study provides insights into the
  security implications of reasoning models and highlights the need for further
  advancements in R1 models' safety to close the gap.
paper_ai_keywords: reasoning models OpenAI-o3 DeepSeek-R1 non-reasoning large language models safety assessment safety benchmarks safety regulations adversarial attacks jailbreaking prompt injection distilled reasoning model safety-aligned base models thinking process