Human-Centered LLM-Agent User Interface: A Position Paper
<|reference_start|>Human-Centered LLM-Agent User Interface: A Position Paper: Large Language Model (LLM) -in-the-loop applications have been shown to effectively interpret the human user's commands, make plans, and operate external tools/systems accordingly. Still, the operation scope of the LLM agent is limited to passively following the user, requiring the user to frame his/her needs with regard to the underlying tools/systems. We note that the potential of an LLM-Agent User Interface (LAUI) is much greater. A user mostly ignorant to the underlying tools/systems should be able to work with a LAUI to discover an emergent workflow. Contrary to the conventional way of designing an explorable GUI to teach the user a predefined set of ways to use the system, in the ideal LAUI, the LLM agent is initialized to be proficient with the system, proactively studies the user and his/her needs, and proposes new interaction schemes to the user. To illustrate LAUI, we present Flute X GPT, a concrete example using an LLM agent, a prompt manager, and a flute-tutoring multi-modal software-hardware system to facilitate the complex, real-time user experience of learning to play the flute.<|reference_end|>
@article{chin2024human-centered,
    title={Human-Centered LLM-Agent User Interface: A Position Paper},
    author={Daniel Chin, Yuxuan Wang, Gus Xia},
    journal={arXiv preprint arXiv:2405.13050},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.13050},
    primaryClass={cs.HC cs.AI}
}