ProcessTBench: An LLM Plan Generation Dataset for Process Mining
<|reference_start|>ProcessTBench: An LLM Plan Generation Dataset for Process Mining: Large Language Models (LLMs) have shown significant promise in plan generation. Yet, existing datasets often lack the complexity needed for advanced tool use scenarios - such as handling paraphrased query statements, supporting multiple languages, and managing actions that can be done in parallel. These scenarios are crucial for evaluating the evolving capabilities of LLMs in real-world applications. Moreover, current datasets don't enable the study of LLMs from a process perspective, particularly in scenarios where understanding typical behaviors and challenges in executing the same process under different conditions or formulations is crucial. To address these gaps, we present the ProcessTBench synthetic dataset, an extension of the TaskBench dataset specifically designed to evaluate LLMs within a process mining framework.<|reference_end|>
@article{redis2024processtbench:,
    title={ProcessTBench: An LLM Plan Generation Dataset for Process Mining},
    author={Andrei Cosmin Redis, Mohammadreza Fani Sani, Bahram Zarrin and Andrea
  Burattin},
    journal={arXiv preprint arXiv:2409.09191},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.09191},
    primaryClass={cs.LG cs.AI cs.ET}
}