title: PromptBench: A Unified Library for Evaluation of Large Language Models
summary: The evaluation of large language models (LLMs) is crucial to assess their
  performance and mitigate potential security risks. In this paper, we introduce
  PromptBench, a unified library to evaluate LLMs. It consists of several key
  components that are easily used and extended by researchers: prompt
  construction, prompt engineering, dataset and model loading, adversarial prompt
  attack, dynamic evaluation protocols, and analysis tools. PromptBench is
  designed to be an open, general, and flexible codebase for research purposes
  that can facilitate original study in creating new benchmarks, deploying
  downstream applications, and designing new evaluation protocols. The code is
  available at: https://github.com/microsoft/promptbench and will be continuously
  supported.
paper_authors._id: 657a66e680141bab5dd86bae
paper_authors.name: Kaijie Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-14T13:35:38.488Z
paper_authors._id: 641fb3733c49fc33cedb4429
paper_authors.avatarUrl: /avatars/3577bca726282ed4b26009f3a2d5e87d.svg
paper_authors.fullname: Kaijie Zhu
paper_authors.type: user
paper_authors.user: March07
paper_authors._id: 657a66e680141bab5dd86baf
paper_authors.name: Qinlin Zhao
paper_authors._id: 657a66e680141bab5dd86bb0
paper_authors.name: Hao Chen
paper_authors._id: 657a66e680141bab5dd86bb1
paper_authors.name: Jindong Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-14T13:36:41.053Z
paper_authors._id: 6204cc0d522e40b4a18d86e2
paper_authors.avatarUrl: /avatars/18daf2de5671e711dc745388dd60569d.svg
paper_authors.fullname: Jindong Wang
paper_authors.type: user
paper_authors.user: jindongwang
paper_authors._id: 657a66e680141bab5dd86bb2
paper_authors.name: Xing Xie
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-14T13:36:48.117Z
paper_authors._id: 64785ca82b3730b6b63b75a2
paper_authors.avatarUrl: /avatars/b0c4f7fb4d4c556118fa6b52d3615429.svg
paper_authors.fullname: Xing Xie
paper_authors.type: user
paper_authors.user: xingustc
paper_title: PromptBench: A Unified Library for Evaluation of Large Language Models
paper_summary: The evaluation of large language models (LLMs) is crucial to assess their
  performance and mitigate potential security risks. In this paper, we introduce
  PromptBench, a unified library to evaluate LLMs. It consists of several key
  components that are easily used and extended by researchers: prompt
  construction, prompt engineering, dataset and model loading, adversarial prompt
  attack, dynamic evaluation protocols, and analysis tools. PromptBench is
  designed to be an open, general, and flexible codebase for research purposes
  that can facilitate original study in creating new benchmarks, deploying
  downstream applications, and designing new evaluation protocols. The code is
  available at: https://github.com/microsoft/promptbench and will be continuously
  supported.
paper_ai_keywords: large context models (LLMs) prompt construction prompt engineering prompt constraint dynamic evaluation protocols analysis tools PromptBench