Designing and Evaluating Dialogue LLMs for Co-Creative Improvised Theatre
<|reference_start|>Designing and Evaluating Dialogue LLMs for Co-Creative Improvised Theatre: Social robotics researchers are increasingly interested in multi-party trained conversational agents. With a growing demand for real-world evaluations, our study presents Large Language Models (LLMs) deployed in a month-long live show at the Edinburgh Festival Fringe. This case study investigates human improvisers co-creating with conversational agents in a professional theatre setting. We explore the technical capabilities and constraints of on-the-spot multi-party dialogue, providing comprehensive insights from both audience and performer experiences with AI on stage. Our human-in-the-loop methodology underlines the challenges of these LLMs in generating context-relevant responses, stressing the user interface's crucial role. Audience feedback indicates an evolving interest for AI-driven live entertainment, direct human-AI interaction, and a diverse range of expectations about AI's conversational competence and utility as a creativity support tool. Human performers express immense enthusiasm, varied satisfaction, and the evolving public opinion highlights mixed emotions about AI's role in arts.<|reference_end|>
@article{branch2024designing,
    title={Designing and Evaluating Dialogue LLMs for Co-Creative Improvised
  Theatre},
    author={Boyd Branch, Piotr Mirowski, Kory Mathewson, Sophia Ppali, Alexandra
  Covaci},
    journal={arXiv preprint arXiv:2405.07111},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.07111},
    primaryClass={cs.CL}
}