"We Need Structured Output": Towards User-centered Constraints on Large Language Model Output
<|reference_start|>"We Need Structured Output": Towards User-centered Constraints on Large Language Model Output: Large language models can produce creative and diverse responses. However, to integrate them into current developer workflows, it is essential to constrain their outputs to follow specific formats or standards. In this work, we surveyed 51 experienced industry professionals to understand the range of scenarios and motivations driving the need for output constraints from a user-centered perspective. We identified 134 concrete use cases for constraints at two levels: low-level, which ensures the output adhere to a structured format and an appropriate length, and high-level, which requires the output to follow semantic and stylistic guidelines without hallucination. Critically, applying output constraints could not only streamline the currently repetitive process of developing, testing, and integrating LLM prompts for developers, but also enhance the user experience of LLM-powered features and applications. We conclude with a discussion on user preferences and needs towards articulating intended constraints for LLMs, alongside an initial design for a constraint prototyping tool.<|reference_end|>
@article{liu2024"we,
    title={"We Need Structured Output": Towards User-centered Constraints on Large
  Language Model Output},
    author={Michael Xieyang Liu, Frederick Liu, Alexander J. Fiannaca, Terry Koo,
  Lucas Dixon, Michael Terry, Carrie J. Cai},
    journal={"We Need Structured Output": Towards User-centered Constraints on
  LLM Output. In Extended Abstracts of the CHI Conference on Human Factors in
  Computing Systems (CHI EA '24), May 11-16, 2024, Honolulu, HI, USA},
    year={2024},
    doi={10.1145/3613905.3650756},
    archivePrefix={arXiv},
    eprint={2404.07362},
    primaryClass={cs.HC}
}