title: Virus: Harmful Fine-tuning Attack for Large Language Models Bypassing
    Guardrail Moderation
summary: Recent research shows that Large Language Models (LLMs) are vulnerable to
  harmful fine-tuning attacks -- models lose their safety alignment ability after
  fine-tuning on a few harmful samples. For risk mitigation, a guardrail is
  typically used to filter out harmful samples before fine-tuning. By designing a
  new red-teaming method, we in this paper show that purely relying on the
  moderation guardrail for data filtration is not reliable. Our proposed attack
  method, dubbed Virus, easily bypasses the guardrail moderation by slightly
  modifying the harmful data. Experimental results show that the harmful data
  optimized by Virus is not detectable by the guardrail with up to 100\% leakage
  ratio, and can simultaneously achieve superior attack performance. Finally, the
  key message we want to convey through this paper is that: it is
  reckless to consider guardrail moderation as a clutch at straws towards harmful
  fine-tuning attack, as it cannot solve the inherent safety issue of the
  pre-trained LLMs. Our code is available at https://github.com/git-disl/Virus
paper_authors._id: 679b1319f87b99a2a7c41e36
paper_authors.name: Tiansheng Huang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-30T08:39:47.548Z
paper_authors._id: 67325283b318faa97f7ae5f7
paper_authors.avatarUrl: /avatars/2f83452768148b323c540c43ad695ee6.svg
paper_authors.fullname: TianshengHuang
paper_authors.type: user
paper_authors.user: TianshengHuang
paper_authors._id: 679b1319f87b99a2a7c41e37
paper_authors.name: Sihao Hu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-30T09:39:58.723Z
paper_authors._id: 6539cab119c3ef6679794706
paper_authors.avatarUrl: /avatars/a88691ff5a547c7a1384edcc615c8209.svg
paper_authors.fullname: Sihao Hu
paper_authors.type: user
paper_authors.user: SihaoHu
paper_authors._id: 679b1319f87b99a2a7c41e38
paper_authors.name: Fatih Ilhan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-30T09:40:06.004Z
paper_authors._id: 647615b995a4dc98e58c24f2
paper_authors.avatarUrl: /avatars/7f73999246526c1aef4d019d5f5595ad.svg
paper_authors.fullname: Fatih Ilhan
paper_authors.type: user
paper_authors.user: tawreos
paper_authors._id: 679b1319f87b99a2a7c41e39
paper_authors.name: Selim Furkan Tekin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-30T09:40:16.339Z
paper_authors._id: 65aae89948c718a57434db6f
paper_authors.avatarUrl: /avatars/6c0fae8dafad9b9265098a9bc3bfc102.svg
paper_authors.fullname: selim tekin
paper_authors.type: user
paper_authors.user: sftekin25
paper_authors._id: 679b1319f87b99a2a7c41e3a
paper_authors.name: Ling Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-30T09:40:37.075Z
paper_authors._id: 65c998005e17dbeaf147db84
paper_authors.avatarUrl: /avatars/6fb47b1e095971b93ff7dcd10369f926.svg
paper_authors.fullname: Ling Liu
paper_authors.type: user
paper_authors.user: ling1119
paper_title: Virus: Harmful Fine-tuning Attack for Large Language Models Bypassing
    Guardrail Moderation
paper_summary: Recent research shows that Large Language Models (LLMs) are vulnerable to
  harmful fine-tuning attacks -- models lose their safety alignment ability after
  fine-tuning on a few harmful samples. For risk mitigation, a guardrail is
  typically used to filter out harmful samples before fine-tuning. By designing a
  new red-teaming method, we in this paper show that purely relying on the
  moderation guardrail for data filtration is not reliable. Our proposed attack
  method, dubbed Virus, easily bypasses the guardrail moderation by slightly
  modifying the harmful data. Experimental results show that the harmful data
  optimized by Virus is not detectable by the guardrail with up to 100\% leakage
  ratio, and can simultaneously achieve superior attack performance. Finally, the
  key message we want to convey through this paper is that: it is
  reckless to consider guardrail moderation as a clutch at straws towards harmful
  fine-tuning attack, as it cannot solve the inherent safety issue of the
  pre-trained LLMs. Our code is available at https://github.com/git-disl/Virus
paper_ai_keywords: large language Models (LLMs) harmful fine-tuning attacks safety alignment ability guardrail moderation guardrail red-teaming method Virus attack method harmful data data filtration leakage ratio attack performance inherent safety issue pre-trained LLMs