title: SafeRAG: Benchmarking Security in Retrieval-Augmented Generation of
    Large Language Model
summary: The indexing-retrieval-generation paradigm of retrieval-augmented generation
  (RAG) has been highly successful in solving knowledge-intensive tasks by
  integrating external knowledge into large language models (LLMs). However, the
  incorporation of external and unverified knowledge increases the vulnerability
  of LLMs because attackers can perform attack tasks by manipulating knowledge.
  In this paper, we introduce a benchmark named SafeRAG designed to evaluate the
  RAG security. First, we classify attack tasks into silver noise, inter-context
  conflict, soft ad, and white Denial-of-Service. Next, we construct RAG security
  evaluation dataset (i.e., SafeRAG dataset) primarily manually for each task. We
  then utilize the SafeRAG dataset to simulate various attack scenarios that RAG
  may encounter. Experiments conducted on 14 representative RAG components
  demonstrate that RAG exhibits significant vulnerability to all attack tasks and
  even the most apparent attack task can easily bypass existing retrievers,
  filters, or advanced LLMs, resulting in the degradation of RAG service quality.
  Code is available at: https://github.com/IAAR-Shanghai/SafeRAG.
paper_authors._id: 67a1bfc314cba2eba6da4b2b
paper_authors.name: Xun Liang
paper_authors._id: 67a1bfc314cba2eba6da4b2c
paper_authors.name: Simin Niu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-04T17:11:40.648Z
paper_authors._id: 66daea8776dbaaa372eabec5
paper_authors.avatarUrl: /avatars/1e5fbe4ff06bb6121c7029253b76b79f.svg
paper_authors.fullname: siminniu
paper_authors.type: user
paper_authors.user: siminniu
paper_authors._id: 67a1bfc314cba2eba6da4b2d
paper_authors.name: Zhiyu Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-04T17:11:52.391Z
paper_authors._id: 661ef43ff65a7cf84e2291e1
paper_authors.avatarUrl: /avatars/cf90dba5934763693c800b3708ce4771.svg
paper_authors.fullname: Zhiyu (Drew) Li
paper_authors.type: user
paper_authors.user: zhiyuli
paper_authors._id: 67a1bfc314cba2eba6da4b2e
paper_authors.name: Sensen Zhang
paper_authors._id: 67a1bfc314cba2eba6da4b2f
paper_authors.name: Hanyu Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-04T09:39:04.452Z
paper_authors._id: 669e0b93c7cb0568dac6e92e
paper_authors.avatarUrl: /avatars/a39ea77d7391f164af8a80f94f85f2ca.svg
paper_authors.fullname: hanyu Wang
paper_authors.type: user
paper_authors.user: UglyToilet
paper_authors._id: 67a1bfc314cba2eba6da4b30
paper_authors.name: Feiyu Xiong
paper_authors._id: 67a1bfc314cba2eba6da4b31
paper_authors.name: Jason Zhaoxin Fan
paper_authors._id: 67a1bfc314cba2eba6da4b32
paper_authors.name: Bo Tang
paper_authors._id: 67a1bfc314cba2eba6da4b33
paper_authors.name: Shichao Song
paper_authors._id: 67a1bfc314cba2eba6da4b34
paper_authors.name: Mengwei Wang
paper_authors._id: 67a1bfc314cba2eba6da4b35
paper_authors.name: Jiawei Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-21T10:00:17.081Z
paper_authors._id: 669e60ee8580d17cb60f8347
paper_authors.avatarUrl: /avatars/37963b833228afe39cc24854c9326670.svg
paper_authors.fullname: yang jiawei
paper_authors.type: user
paper_authors.user: Dany-0
paper_title: SafeRAG: Benchmarking Security in Retrieval-Augmented Generation of
    Large Language Model
paper_summary: The indexing-retrieval-generation paradigm of retrieval-augmented generation
  (RAG) has been highly successful in solving knowledge-intensive tasks by
  integrating external knowledge into large language models (LLMs). However, the
  incorporation of external and unverified knowledge increases the vulnerability
  of LLMs because attackers can perform attack tasks by manipulating knowledge.
  In this paper, we introduce a benchmark named SafeRAG designed to evaluate the
  RAG security. First, we classify attack tasks into silver noise, inter-context
  conflict, soft ad, and white Denial-of-Service. Next, we construct RAG security
  evaluation dataset (i.e., SafeRAG dataset) primarily manually for each task. We
  then utilize the SafeRAG dataset to simulate various attack scenarios that RAG
  may encounter. Experiments conducted on 14 representative RAG components
  demonstrate that RAG exhibits significant vulnerability to all attack tasks and
  even the most apparent attack task can easily bypass existing retrievers,
  filters, or advanced LLMs, resulting in the degradation of RAG service quality.
  Code is available at: https://github.com/IAAR-Shanghai/SafeRAG.
paper_ai_summary: The SafeRAG benchmark evaluates the security of RAG systems by simulating various attack scenarios, revealing significant vulnerabilities in RAG components.
paper_ai_keywords: retrieval-augmented generation RAG large language models external knowledge silver noise inter-context conflict soft ad white Denial-of-Service RAG security evaluation dataset retrievers filters