Conformal Intent Classification and Clarification for Fast and Accurate Intent Recognition
<|reference_start|>Conformal Intent Classification and Clarification for Fast and Accurate Intent Recognition: We present Conformal Intent Classification and Clarification (CICC), a framework for fast and accurate intent classification for task-oriented dialogue systems. The framework turns heuristic uncertainty scores of any intent classifier into a clarification question that is guaranteed to contain the true intent at a pre-defined confidence level. By disambiguating between a small number of likely intents, the user query can be resolved quickly and accurately. Additionally, we propose to augment the framework for out-of-scope detection. In a comparative evaluation using seven intent recognition datasets we find that CICC generates small clarification questions and is capable of out-of-scope detection. CICC can help practitioners and researchers substantially in improving the user experience of dialogue agents with specific clarification questions.<|reference_end|>
@article{hengst2024conformal,
    title={Conformal Intent Classification and Clarification for Fast and Accurate
  Intent Recognition},
    author={Floris den Hengst, Ralf Wolter, Patrick Altmeyer and Arda Kaygan},
    journal={arXiv preprint arXiv:2403.18973},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.18973},
    primaryClass={cs.CL}
}