title: Exploiting Novel GPT-4 APIs
summary: Language model attacks typically assume one of two extreme threat models:
  full white-box access to model weights, or black-box access limited to a text
  generation API. However, real-world APIs are often more flexible than just text
  generation: these APIs expose ``gray-box'' access leading to new threat
  vectors. To explore this, we red-team three new functionalities exposed in the
  GPT-4 APIs: fine-tuning, function calling and knowledge retrieval. We find that
  fine-tuning a model on as few as 15 harmful examples or 100 benign examples can
  remove core safeguards from GPT-4, enabling a range of harmful outputs.
  Furthermore, we find that GPT-4 Assistants readily divulge the function call
  schema and can be made to execute arbitrary function calls. Finally, we find
  that knowledge retrieval can be hijacked by injecting instructions into
  retrieval documents. These vulnerabilities highlight that any additions to the
  functionality exposed by an API can create new vulnerabilities.
paper_authors._id: 658a40c5d1331d552b250e8a
paper_authors.name: Kellin Pelrine
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-29T15:22:36.818Z
paper_authors._id: 6596269ef8f4854a7781dba1
paper_authors.avatarUrl: /avatars/680856920131c63c0ad6a67ed7db1db8.svg
paper_authors.fullname: Kellin Pelrine
paper_authors.type: user
paper_authors.user: kellinpelrine
paper_authors._id: 658a40c5d1331d552b250e8b
paper_authors.name: Mohammad Taufeeque
paper_authors._id: 658a40c5d1331d552b250e8c
paper_authors.name: Michał Zając
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-01-25T19:04:47.257Z
paper_authors._id: 65b2b0884f699ea319d2570f
paper_authors.avatarUrl: /avatars/42620ac19e28f1a6a11a9ed270a0e955.svg
paper_authors.fullname: Michał Zając
paper_authors.type: user
paper_authors.user: michalzajac
paper_authors._id: 658a40c5d1331d552b250e8d
paper_authors.name: Euan McLean
paper_authors._id: 658a40c5d1331d552b250e8e
paper_authors.name: Adam Gleave
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-12T08:06:45.599Z
paper_authors._id: 6294db83b9d93a419906c9bb
paper_authors.avatarUrl: /avatars/ccd716242a7a1b0024bcf92cb6134a8f.svg
paper_authors.fullname: Adam Gleave
paper_authors.type: user
paper_authors.user: AdamGleave
paper_title: Exploiting Novel GPT-4 APIs
paper_summary: Language model attacks typically assume one of two extreme threat models:
  full white-box access to model weights, or black-box access limited to a text
  generation API. However, real-world APIs are often more flexible than just text
  generation: these APIs expose ``gray-box'' access leading to new threat
  vectors. To explore this, we red-team three new functionalities exposed in the
  GPT-4 APIs: fine-tuning, function calling and knowledge retrieval. We find that
  fine-tuning a model on as few as 15 harmful examples or 100 benign examples can
  remove core safeguards from GPT-4, enabling a range of harmful outputs.
  Furthermore, we find that GPT-4 Assistants readily divulge the function call
  schema and can be made to execute arbitrary function calls. Finally, we find
  that knowledge retrieval can be hijacked by injecting instructions into
  retrieval documents. These vulnerabilities highlight that any additions to the
  functionality exposed by an API can create new vulnerabilities.
paper_ai_keywords: fine-tuning function calling knowledge retrieval harmful outputs function call schema