Safe Generative Chats in a WhatsApp Intelligent Tutoring System
<|reference_start|>Safe Generative Chats in a WhatsApp Intelligent Tutoring System: Large language models (LLMs) are flexible, personalizable, and available, which makes their use within Intelligent Tutoring Systems (ITSs) appealing. However, that flexibility creates risks: inaccuracies, harmful content, and non-curricular material. Ethically deploying LLM-backed ITS systems requires designing safeguards that ensure positive experiences for students. We describe the design of a conversational system integrated into an ITS, and our experience evaluating its safety with red-teaming, an in-classroom usability test, and field deployment. We present empirical data from more than 8,000 student conversations with this system, finding that GPT-3.5 rarely generates inappropriate messages. Comparatively more common is inappropriate messages from students, which prompts us to reason about safeguarding as a content moderation and classroom management problem. The student interaction behaviors we observe provide implications for designers - to focus on student inputs as a content moderation problem - and implications for researchers - to focus on subtle forms of bad content.<|reference_end|>
@article{levonian2024safe,
    title={Safe Generative Chats in a WhatsApp Intelligent Tutoring System},
    author={Zachary Levonian, Owen Henkel},
    journal={arXiv preprint arXiv:2407.04915},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.04915},
    primaryClass={cs.HC}
}