content.raw: Stealing Part of a Production Language Model
content.type: text
content.value: Stealing Part of a Production Language Model
content.type: new_line
content.type: new_line
content.label: Stealing Part of a Production Language Model (2403.06634)
content.raw: https://huggingface.co/papers/2403.06634
content.id: 2403.06634
content.type: paper
content.type: resource
content.url: https://huggingface.co/papers/2403.06634
content.type: new_line
content.type: new_line
content.raw: We introduce the first model-stealing attack that extracts precise, nontrivial information from black-box production language models like OpenAI's ChatGPT or Google's PaLM-2. Specifically, our attack recovers the embedding projection layer (up to symmetries) of a transformer model, given typical API access. For under \20 USD, our attack extracts the entire projection matrix of OpenAI's Ada and Babbage language models. We thereby confirm, for the first time, that these black-box models have a hidden dimension of 1024 and 2048, respectively. We also recover the exact hidden dimension size of the gpt-3.5-turbo model, and estimate it would cost under 2,000 in queries to recover the entire projection matrix. We conclude with potential defenses and mitigations, and discuss the implications of possible future work that could extend our attack.
content.type: text
content.value: We introduce the first model-stealing attack that extracts precise, nontrivial information from black-box production language models like OpenAI's ChatGPT or Google's PaLM-2. Specifically, our attack recovers the embedding projection layer (up to symmetries) of a transformer model, given typical API access. For under \20 USD, our attack extracts the entire projection matrix of OpenAI's Ada and Babbage language models. We thereby confirm, for the first time, that these black-box models have a hidden dimension of 1024 and 2048, respectively. We also recover the exact hidden dimension size of the gpt-3.5-turbo model, and estimate it would cost under 2,000 in queries to recover the entire projection matrix. We conclude with potential defenses and mitigations, and discuss the implications of possible future work that could extend our attack.
rawContent: Stealing Part of a Production Language Model
  
  https://huggingface.co/papers/2403.06634
  
  We introduce the first model-stealing attack that extracts precise, nontrivial information from black-box production language models like OpenAI's ChatGPT or Google's PaLM-2. Specifically, our attack recovers the embedding projection layer (up to symmetries) of a transformer model, given typical API access. For under \20 USD, our attack extracts the entire projection matrix of OpenAI's Ada and Babbage language models. We thereby confirm, for the first time, that these black-box models have a hidden dimension of 1024 and 2048, respectively. We also recover the exact hidden dimension size of the gpt-3.5-turbo model, and estimate it would cost under 2,000 in queries to recover the entire projection matrix. We conclude with potential defenses and mitigations, and discuss the implications of possible future work that could extend our attack.
reactions.reaction: 👍
reactions.users: ajibawa-2023 taufiqdp arnavgrg adamelliotfields evdcush abdullahalzubaer bootsbootsboots philipp-zettl
reactions.reaction: 🤯
reactions.users: adamelliotfields CristianJD diwank
reactions.reaction: 🧠
reactions.users: merve
reactions.reaction: 🔥
reactions.users: diwank
commentators._id: 6141a88b3a0ec78603c9e784
commentators.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6141a88b3a0ec78603c9e784/DJsxSmWV39M33JFheLobC.jpeg
commentators.fullname: Merve Noyan
commentators.name: merve
commentators.type: user