PROMPTEVALS: A Dataset of Assertions and Guardrails for Custom
  Production Large Language Model Pipelines
Large language models (LLMs) are increasingly deployed in specialized
production data processing pipelines across diverse domains -- such as finance,
marketing, and e-commerce. However, when running them in production across many
inputs, they often fail to follow instructions or meet developer expectations.
To improve reliability in these applications, creating assertions or guardrails
for LLM outputs to run alongside the pipelines is essential. Yet, determining
the right set of assertions that capture developer requirements for a task is
challenging. In this paper, we introduce PROMPTEVALS, a dataset of 2087 LLM
pipeline prompts with 12623 corresponding assertion criteria, sourced from
developers using our open-source LLM pipeline tools. This dataset is 5x larger
than previous collections. Using a hold-out test split of PROMPTEVALS as a
benchmark, we evaluated closed- and open-source models in generating relevant
assertions. Notably, our fine-tuned Mistral and Llama 3 models outperform
GPT-4o by 20.93% on average, offering both reduced latency and improved
performance. We believe our dataset can spur further research in LLM
reliability, alignment, and prompt engineering.
68074e6b6a30fb43d9503b11
Reya Vir
extracted_confirmed
2025-04-22T16:05:27.348Z
65d28835f8ab7fc6697d854b
/avatars/c0077cbf930c5eee11cf4c2eace6f502.svg
Reya Vir
user
reyavir
68074e6b6a30fb43d9503b12
Shreya Shankar
extracted_pending
2025-04-22T08:08:12.241Z
63f53f733aa49d8cb97b15e8
/avatars/2b23950c845a1ad3b4b82b437ea52562.svg
Shreya Shankar
user
shreyashankar
68074e6b6a30fb43d9503b13
Harrison Chase
68074e6b6a30fb43d9503b14
Will Fu-Hinthorn
68074e6b6a30fb43d9503b15
Aditya Parameswaran
PROMPTEVALS: A Dataset of Assertions and Guardrails for Custom
  Production Large Language Model Pipelines
Large language models (LLMs) are increasingly deployed in specialized
production data processing pipelines across diverse domains -- such as finance,
marketing, and e-commerce. However, when running them in production across many
inputs, they often fail to follow instructions or meet developer expectations.
To improve reliability in these applications, creating assertions or guardrails
for LLM outputs to run alongside the pipelines is essential. Yet, determining
the right set of assertions that capture developer requirements for a task is
challenging. In this paper, we introduce PROMPTEVALS, a dataset of 2087 LLM
pipeline prompts with 12623 corresponding assertion criteria, sourced from
developers using our open-source LLM pipeline tools. This dataset is 5x larger
than previous collections. Using a hold-out test split of PROMPTEVALS as a
benchmark, we evaluated closed- and open-source models in generating relevant
assertions. Notably, our fine-tuned Mistral and Llama 3 models outperform
GPT-4o by 20.93% on average, offering both reduced latency and improved
performance. We believe our dataset can spur further research in LLM
reliability, alignment, and prompt engineering.
The PROMPTEVALS dataset contains a large number of prompts and assertion criteria for assessing the reliability of LLMs in production, demonstrating improved assertion generation performance with fine-tuned models compared to GPT-4o.
PROMPTEVALS large language models (LLMs) assertions guardrails prompt engineering Mistral Llama 3 GPT-4o LLM pipeline tools