PromptBench: A Unified Library for Evaluation of Large Language Models
The evaluation of large language models (LLMs) is crucial to assess their
performance and mitigate potential security risks. In this paper, we introduce
PromptBench, a unified library to evaluate LLMs. It consists of several key
components that are easily used and extended by researchers: prompt
construction, prompt engineering, dataset and model loading, adversarial prompt
attack, dynamic evaluation protocols, and analysis tools. PromptBench is
designed to be an open, general, and flexible codebase for research purposes
that can facilitate original study in creating new benchmarks, deploying
downstream applications, and designing new evaluation protocols. The code is
available at: https://github.com/microsoft/promptbench and will be continuously
supported.
657a66e680141bab5dd86bae
Kaijie Zhu
admin_assigned
2023-12-14T13:35:38.488Z
641fb3733c49fc33cedb4429
/avatars/3577bca726282ed4b26009f3a2d5e87d.svg
Kaijie Zhu
user
March07
657a66e680141bab5dd86baf
Qinlin Zhao
657a66e680141bab5dd86bb0
Hao Chen
657a66e680141bab5dd86bb1
Jindong Wang
admin_assigned
2023-12-14T13:36:41.053Z
6204cc0d522e40b4a18d86e2
/avatars/18daf2de5671e711dc745388dd60569d.svg
Jindong Wang
user
jindongwang
657a66e680141bab5dd86bb2
Xing Xie
admin_assigned
2023-12-14T13:36:48.117Z
64785ca82b3730b6b63b75a2
/avatars/b0c4f7fb4d4c556118fa6b52d3615429.svg
Xing Xie
user
xingustc
PromptBench: A Unified Library for Evaluation of Large Language Models
The evaluation of large language models (LLMs) is crucial to assess their
performance and mitigate potential security risks. In this paper, we introduce
PromptBench, a unified library to evaluate LLMs. It consists of several key
components that are easily used and extended by researchers: prompt
construction, prompt engineering, dataset and model loading, adversarial prompt
attack, dynamic evaluation protocols, and analysis tools. PromptBench is
designed to be an open, general, and flexible codebase for research purposes
that can facilitate original study in creating new benchmarks, deploying
downstream applications, and designing new evaluation protocols. The code is
available at: https://github.com/microsoft/promptbench and will be continuously
supported.
PromptBench is a unified library designed to evaluate large language models with key components for prompt engineering, adversarial attacks, and flexible evaluation protocols.
prompt construction prompt engineering adversarial prompt attack dynamic evaluation protocols analysis tools PromptBench