content.raw: Mixture of experts: beware 🛡️⚔️
content.type: text
content.value: Mixture of experts: beware 🛡️⚔️
content.type: new_line
content.type: new_line
content.raw: New paper by DeepMind: Buffer Overflow in MoE 
content.type: text
content.value: New paper by DeepMind: Buffer Overflow in MoE 
content.label: Buffer Overflow in Mixture of Experts (2402.05526)
content.raw: https://huggingface.co/papers/2402.05526
content.id: 2402.05526
content.type: paper
content.type: resource
content.url: https://huggingface.co/papers/2402.05526
content.type: new_line
content.type: new_line
content.raw: The paper shows an adversarial attack strategy in which a user sends malicious queries that can affect the output of other user queries from the same batch. 
content.type: text
content.value: The paper shows an adversarial attack strategy in which a user sends malicious queries that can affect the output of other user queries from the same batch. 
content.type: new_line
content.type: new_line
content.raw: So if in the same batch we have 
content.type: text
content.value: So if in the same batch we have 
content.type: new_line
content.raw: - User A benign query
content.type: text
content.value: - User A benign query
content.type: new_line
content.raw: - User B malicious query
content.type: text
content.value: - User B malicious query
content.type: new_line
content.raw: The response for A might be altered!😱
content.type: text
content.value: The response for A might be altered!😱
content.type: new_line
content.type: new_line
content.raw: How is this possible?
content.type: text
content.value: How is this possible?
content.type: new_line
content.raw: One approach is to fill the token buffers with adversarial data, hence forcing the gating to use the non-ideal experts or to entirely drop the bening tokens (in the case of finite limit size). 
content.type: text
content.value: One approach is to fill the token buffers with adversarial data, hence forcing the gating to use the non-ideal experts or to entirely drop the bening tokens (in the case of finite limit size). 
content.type: new_line
content.type: new_line
content.raw: This assumes that the adversary can use the model as a black-box but can observe the logit outputs + ensure that the data is always grouped in the same batch. 
content.type: text
content.value: This assumes that the adversary can use the model as a black-box but can observe the logit outputs + ensure that the data is always grouped in the same batch. 
content.type: new_line
content.type: new_line
content.raw: How to mitigate this?
content.type: text
content.value: How to mitigate this?
content.type: new_line
content.raw: - Randomize batch order (and even run twice if some queries are very sensitive)
content.type: text
content.value: - Randomize batch order (and even run twice if some queries are very sensitive)
content.type: new_line
content.raw: - Use a large capacity slack
content.type: text
content.value: - Use a large capacity slack
content.type: new_line
content.raw: - Sample from gate weights instead of top-k (not great IMO, as that require more memory for inference)
content.type: text
content.value: - Sample from gate weights instead of top-k (not great IMO, as that require more memory for inference)
content.type: new_line
content.type: new_line
content.raw: Very cool paper!!
content.type: text
content.value: Very cool paper!!
rawContent: Mixture of experts: beware 🛡️⚔️
  
  New paper by DeepMind: Buffer Overflow in MoE https://huggingface.co/papers/2402.05526
  
  The paper shows an adversarial attack strategy in which a user sends malicious queries that can affect the output of other user queries from the same batch. 
  
  So if in the same batch we have 
  - User A benign query
  - User B malicious query
  The response for A might be altered!😱
  
  How is this possible?
  One approach is to fill the token buffers with adversarial data, hence forcing the gating to use the non-ideal experts or to entirely drop the bening tokens (in the case of finite limit size). 
  
  This assumes that the adversary can use the model as a black-box but can observe the logit outputs + ensure that the data is always grouped in the same batch. 
  
  How to mitigate this?
  - Randomize batch order (and even run twice if some queries are very sensitive)
  - Use a large capacity slack
  - Sample from gate weights instead of top-k (not great IMO, as that require more memory for inference)
  
  Very cool paper!!
reactions.reaction: 👍
reactions.users: NickyNicky julien-c macadeliccc ivanfioravanti clem samusenps victor JosephGesnouin pratjz blancsw marinaretik juancopi81 Dlbk djmcflush EmelMk surfiend Yasbok 0xe69756
reactions.reaction: ❤️
reactions.users: macadeliccc clem samusenps JosephGesnouin blancsw santiviquez Dlbk abunchofrandomwords 0xe69756
reactions.reaction: 🤯
reactions.users: meefs blancsw davanstrien Dlbk Yhyu13 Epiculous
reactions.reaction: 🤝
reactions.users: Dlbk Epiculous
commentators._id: 64438bcb1bc692d87b237c04
commentators.avatarUrl: /avatars/19f9eeb9281b47b34f68c312092ca468.svg
commentators.fullname: Yu
commentators.name: Yhyu13
commentators.type: user
commentators._id: 64adfd277b5ff762771e4571
commentators.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64adfd277b5ff762771e4571/HVmAmGOtBPYd72492XGar.png
commentators.fullname: Epiculous
commentators.name: Epiculous
commentators.type: user