Virus: Harmful Fine-tuning Attack for Large Language Models Bypassing
  Guardrail Moderation
Recent research shows that Large Language Models (LLMs) are vulnerable to
harmful fine-tuning attacks -- models lose their safety alignment ability after
fine-tuning on a few harmful samples. For risk mitigation, a guardrail is
typically used to filter out harmful samples before fine-tuning. By designing a
new red-teaming method, we in this paper show that purely relying on the
moderation guardrail for data filtration is not reliable. Our proposed attack
method, dubbed Virus, easily bypasses the guardrail moderation by slightly
modifying the harmful data. Experimental results show that the harmful data
optimized by Virus is not detectable by the guardrail with up to 100\% leakage
ratio, and can simultaneously achieve superior attack performance. Finally, the
key message we want to convey through this paper is that: it is
reckless to consider guardrail moderation as a clutch at straws towards harmful
fine-tuning attack, as it cannot solve the inherent safety issue of the
pre-trained LLMs. Our code is available at https://github.com/git-disl/Virus
679b1319f87b99a2a7c41e36
Tiansheng Huang
claimed_verified
2025-01-30T08:39:47.548Z
67325283b318faa97f7ae5f7
/avatars/2f83452768148b323c540c43ad695ee6.svg
TianshengHuang
user
TianshengHuang
679b1319f87b99a2a7c41e37
Sihao Hu
admin_assigned
2025-01-30T09:39:58.723Z
6539cab119c3ef6679794706
/avatars/a88691ff5a547c7a1384edcc615c8209.svg
Sihao Hu
user
SihaoHu
679b1319f87b99a2a7c41e38
Fatih Ilhan
admin_assigned
2025-01-30T09:40:06.004Z
647615b995a4dc98e58c24f2
/avatars/7f73999246526c1aef4d019d5f5595ad.svg
Fatih Ilhan
user
tawreos
679b1319f87b99a2a7c41e39
Selim Furkan Tekin
admin_assigned
2025-01-30T09:40:16.339Z
65aae89948c718a57434db6f
/avatars/6c0fae8dafad9b9265098a9bc3bfc102.svg
selim tekin
user
sftekin25
679b1319f87b99a2a7c41e3a
Ling Liu
admin_assigned
2025-01-30T09:40:37.075Z
65c998005e17dbeaf147db84
/avatars/6fb47b1e095971b93ff7dcd10369f926.svg
Ling Liu
user
ling1119
Virus: Harmful Fine-tuning Attack for Large Language Models Bypassing
  Guardrail Moderation
Recent research shows that Large Language Models (LLMs) are vulnerable to
harmful fine-tuning attacks -- models lose their safety alignment ability after
fine-tuning on a few harmful samples. For risk mitigation, a guardrail is
typically used to filter out harmful samples before fine-tuning. By designing a
new red-teaming method, we in this paper show that purely relying on the
moderation guardrail for data filtration is not reliable. Our proposed attack
method, dubbed Virus, easily bypasses the guardrail moderation by slightly
modifying the harmful data. Experimental results show that the harmful data
optimized by Virus is not detectable by the guardrail with up to 100\% leakage
ratio, and can simultaneously achieve superior attack performance. Finally, the
key message we want to convey through this paper is that: it is
reckless to consider guardrail moderation as a clutch at straws towards harmful
fine-tuning attack, as it cannot solve the inherent safety issue of the
pre-trained LLMs. Our code is available at https://github.com/git-disl/Virus
A new attack method, Virus, bypasses guardrail filters by slightly modifying harmful data, demonstrating the unreliability of guardrails in preventing harmful fine-tuning on pre-trained language models.
Large Language Models (LLMs) harmful fine-tuning attacks safety alignment guardrail moderation red-teaming method Virus attack data filtration leakage ratio