  We present an extensible user simulation toolkit to facilitate automatic
evaluation of conversational recommender systems. It builds on an established
agenda-based approach and extends it with several novel elements, including
user satisfaction prediction, persona and context modeling, and conditional
natural language generation. We showcase the toolkit with a pre-existing movie
recommender system and demonstrate its ability to simulate dialogues that mimic
real conversations, while requiring only a handful of manually annotated
dialogues as training data.

Information Retrieval