title: PROMPTEVALS: A Dataset of Assertions and Guardrails for Custom
    Production Large Language Model Pipelines
summary: Large language models (LLMs) are increasingly deployed in specialized
  production data processing pipelines across diverse domains -- such as finance,
  marketing, and e-commerce. However, when running them in production across many
  inputs, they often fail to follow instructions or meet developer expectations.
  To improve reliability in these applications, creating assertions or guardrails
  for LLM outputs to run alongside the pipelines is essential. Yet, determining
  the right set of assertions that capture developer requirements for a task is
  challenging. In this paper, we introduce PROMPTEVALS, a dataset of 2087 LLM
  pipeline prompts with 12623 corresponding assertion criteria, sourced from
  developers using our open-source LLM pipeline tools. This dataset is 5x larger
  than previous collections. Using a hold-out test split of PROMPTEVALS as a
  benchmark, we evaluated closed- and open-source models in generating relevant
  assertions. Notably, our fine-tuned Mistral and Llama 3 models outperform
  GPT-4o by 20.93% on average, offering both reduced latency and improved
  performance. We believe our dataset can spur further research in LLM
  reliability, alignment, and prompt engineering.
paper_authors._id: 68074e6b6a30fb43d9503b11
paper_authors.name: Reya Vir
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-04-22T16:05:27.348Z
paper_authors._id: 65d28835f8ab7fc6697d854b
paper_authors.avatarUrl: /avatars/c0077cbf930c5eee11cf4c2eace6f502.svg
paper_authors.fullname: Reya Vir
paper_authors.type: user
paper_authors.user: reyavir
paper_authors._id: 68074e6b6a30fb43d9503b12
paper_authors.name: Shreya Shankar
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-04-22T08:08:12.241Z
paper_authors._id: 63f53f733aa49d8cb97b15e8
paper_authors.avatarUrl: /avatars/2b23950c845a1ad3b4b82b437ea52562.svg
paper_authors.fullname: Shreya Shankar
paper_authors.type: user
paper_authors.user: shreyashankar
paper_authors._id: 68074e6b6a30fb43d9503b13
paper_authors.name: Harrison Chase
paper_authors._id: 68074e6b6a30fb43d9503b14
paper_authors.name: Will Fu-Hinthorn
paper_authors._id: 68074e6b6a30fb43d9503b15
paper_authors.name: Aditya Parameswaran
paper_title: PROMPTEVALS: A Dataset of Assertions and Guardrails for Custom
    Production Large Language Model Pipelines
paper_summary: Large language models (LLMs) are increasingly deployed in specialized
  production data processing pipelines across diverse domains -- such as finance,
  marketing, and e-commerce. However, when running them in production across many
  inputs, they often fail to follow instructions or meet developer expectations.
  To improve reliability in these applications, creating assertions or guardrails
  for LLM outputs to run alongside the pipelines is essential. Yet, determining
  the right set of assertions that capture developer requirements for a task is
  challenging. In this paper, we introduce PROMPTEVALS, a dataset of 2087 LLM
  pipeline prompts with 12623 corresponding assertion criteria, sourced from
  developers using our open-source LLM pipeline tools. This dataset is 5x larger
  than previous collections. Using a hold-out test split of PROMPTEVALS as a
  benchmark, we evaluated closed- and open-source models in generating relevant
  assertions. Notably, our fine-tuned Mistral and Llama 3 models outperform
  GPT-4o by 20.93% on average, offering both reduced latency and improved
  performance. We believe our dataset can spur further research in LLM
  reliability, alignment, and prompt engineering.
paper_ai_summary: The PROMPTEVALS dataset contains a large number of prompts and assertion criteria for assessing the reliability of LLMs in production, demonstrating improved assertion generation performance with fine-tuned models compared to GPT-4o.
paper_ai_keywords: PROMPTEVALS large language models (LLMs) assertions guardrails prompt engineering Mistral Llama 3 GPT-4o LLM pipeline tools