  Open-source Large Language Models (LLMs) have recently gained popularity
because of their comparable performance to proprietary LLMs. To efficiently
fulfill domain-specialized tasks, open-source LLMs can be refined, without
expensive accelerators, using low-rank adapters. However, it is still unknown
whether low-rank adapters can be exploited to control LLMs. To address this
gap, we demonstrate that an infected adapter can induce, on specific triggers,
an LLM to output content defined by an adversary and to even maliciously use
tools. To train a Trojan adapter, we propose two novel attacks, POLISHED and
FUSION, that improve over prior approaches. POLISHED uses LLM-enhanced
paraphrasing to polish benchmark poisoned datasets. In contrast, in the absence
of a dataset, FUSION leverages an over-poisoning procedure to transform a
benign adaptor. Our experiments validate that our attacks provide higher attack
effectiveness than the baseline and, for the purpose of attracting downloads,
preserves or improves the adapter's utility. Finally, we provide two case
studies to demonstrate that the Trojan adapter can lead a LLM-powered
autonomous agent to execute unintended scripts or send phishing emails. Our
novel attacks represent the first study of supply chain threats for LLMs
through the lens of Trojan plugins.

Cryptography and Security