What You Say = What You Want? Teaching Humans to Articulate Requirements for LLMs
<|reference_start|>What You Say = What You Want? Teaching Humans to Articulate Requirements for LLMs: Prompting ChatGPT to achieve complex goals (e.g., creating a customer support chatbot) often demands meticulous prompt engineering, including aspects like fluent writing and chain-of-thought techniques. While emerging prompt optimizers can automatically refine many of these aspects, we argue that clearly conveying customized requirements (e.g., how to handle diverse inputs) remains a human-centric challenge. In this work, we introduce Requirement-Oriented Prompt Engineering (ROPE), a paradigm that focuses human attention on generating clear, complete requirements during prompting. We implement ROPE through an assessment and training suite that provides deliberate practice with LLM-generated feedback. In a study with 30 novices, we show that requirement-focused training doubles novices' prompting performance, significantly outperforming conventional prompt engineering training and prompt optimization. We also demonstrate that high-quality LLM outputs are directly tied to the quality of input requirements. Our work paves the way for more effective task delegation in human-LLM collaborative prompting.<|reference_end|>
@article{ma2024what,
    title={What You Say = What You Want? Teaching Humans to Articulate Requirements
  for LLMs},
    author={Qianou Ma, Weirui Peng, Hua Shen, Kenneth Koedinger, Tongshuang Wu},
    journal={arXiv preprint arXiv:2409.08775},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.08775},
    primaryClass={cs.HC cs.AI}
}