title: Stealing User Prompts from Mixture of Experts
summary: Mixture-of-Experts (MoE) models improve the efficiency and scalability of
  dense language models by routing each token to a small number of experts in
  each layer. In this paper, we show how an adversary that can arrange for their
  queries to appear in the same batch of examples as a victim's queries can
  exploit Expert-Choice-Routing to fully disclose a victim's prompt. We
  successfully demonstrate the effectiveness of this attack on a two-layer
  Mixtral model, exploiting the tie-handling behavior of the torch.topk CUDA
  implementation. Our results show that we can extract the entire prompt using
  O({VM}^2) queries (with vocabulary size V and prompt length M) or 100
  queries on average per token in the setting we consider. This is the first
  attack to exploit architectural flaws for the purpose of extracting user
  prompts, introducing a new class of LLM vulnerabilities.
paper_authors._id: 672340ae85f9fe93ec5b20be
paper_authors.name: Itay Yona
paper_authors._id: 672340ae85f9fe93ec5b20bf
paper_authors.name: Ilia Shumailov
paper_authors._id: 672340ae85f9fe93ec5b20c0
paper_authors.name: Jamie Hayes
paper_authors._id: 672340ae85f9fe93ec5b20c1
paper_authors.name: Nicholas Carlini
paper_title: Stealing User Prompts from Mixture of Experts
paper_summary: Mixture-of-Experts (MoE) models improve the efficiency and scalability of
  dense language models by routing each token to a small number of experts in
  each layer. In this paper, we show how an adversary that can arrange for their
  queries to appear in the same batch of examples as a victim's queries can
  exploit Expert-Choice-Routing to fully disclose a victim's prompt. We
  successfully demonstrate the effectiveness of this attack on a two-layer
  Mixtral model, exploiting the tie-handling behavior of the torch.topk CUDA
  implementation. Our results show that we can extract the entire prompt using
  O({VM}^2) queries (with vocabulary size V and prompt length M) or 100
  queries on average per token in the setting we consider. This is the first
  attack to exploit architectural flaws for the purpose of extracting user
  prompts, introducing a new class of LLM vulnerabilities.
paper_ai_summary: Attackers can exploit architectural flaws in Mixture-of-Experts models to extract user prompts by manipulating query batches in a specific manner.
paper_ai_keywords: Mixture-of-Experts Expert-Choice-Routing MoE tie-handling behavior torch.topk CUDA