PROC2PDDL: Open-Domain Planning Representations from Texts
<|reference_start|>PROC2PDDL: Open-Domain Planning Representations from Texts: Planning in a text-based environment continues to be a major challenge for AI systems. Recent approaches have used language models to predict a planning domain definition (e.g., PDDL) but have only been evaluated in closed-domain simulated environments. To address this, we present Proc2PDDL , the first dataset containing open-domain procedural texts paired with expert-annotated PDDL representations. Using this dataset, we evaluate state-of-the-art models on defining the preconditions and effects of actions. We show that Proc2PDDL is highly challenging, with GPT-3.5's success rate close to 0% and GPT-4's around 35%. Our analysis shows both syntactic and semantic errors, indicating LMs' deficiency in both generating domain-specific prgorams and reasoning about events. We hope this analysis and dataset helps future progress towards integrating the best of LMs and formal planning.<|reference_end|>
@article{zhang2024proc2pddl:,
    title={PROC2PDDL: Open-Domain Planning Representations from Texts},
    author={Tianyi Zhang, Li Zhang, Zhaoyi Hou, Ziyu Wang, Yuling Gu, Peter Clark,
  Chris Callison-Burch, Niket Tandon},
    journal={arXiv preprint arXiv:2403.00092},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.00092},
    primaryClass={cs.CL}
}