Identifying Breakdowns in Conversational Recommender Systems using User Simulation
<|reference_start|>Identifying Breakdowns in Conversational Recommender Systems using User Simulation: We present a methodology to systematically test conversational recommender systems with regards to conversational breakdowns. It involves examining conversations generated between the system and simulated users for a set of pre-defined breakdown types, extracting responsible conversational paths, and characterizing them in terms of the underlying dialogue intents. User simulation offers the advantages of simplicity, cost-effectiveness, and time efficiency for obtaining conversations where potential breakdowns can be identified. The proposed methodology can be used as diagnostic tool as well as a development tool to improve conversational recommendation systems. We apply our methodology in a case study with an existing conversational recommender system and user simulator, demonstrating that with just a few iterations, we can make the system more robust to conversational breakdowns.<|reference_end|>
@article{bernard2024identifying,
    title={Identifying Breakdowns in Conversational Recommender Systems using User
  Simulation},
    author={Nolwenn Bernard and Krisztian Balog},
    journal={arXiv preprint arXiv:2405.14249},
    year={2024},
    doi={10.1145/3640794.3665539},
    archivePrefix={arXiv},
    eprint={2405.14249},
    primaryClass={cs.IR}
}