Evaluation and Continual Improvement for an Enterprise AI Assistant
<|reference_start|>Evaluation and Continual Improvement for an Enterprise AI Assistant: The development of conversational AI assistants is an iterative process with multiple components. As such, the evaluation and continual improvement of these assistants is a complex and multifaceted problem. This paper introduces the challenges in evaluating and improving a generative AI assistant for enterprises, which is under active development, and how we address these challenges. We also share preliminary results and discuss lessons learned.<|reference_end|>
@article{maharaj2024evaluation,
    title={Evaluation and Continual Improvement for an Enterprise AI Assistant},
    author={Akash V. Maharaj, Kun Qian, Uttaran Bhattacharya, Sally Fang, Horia
  Galatanu, Manas Garg, Rachel Hanessian, Nishant Kapoor, Ken Russell,
  Shivakumar Vaithyanathan, and Yunyao Li},
    journal={arXiv preprint arXiv:2407.12003},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.12003},
    primaryClass={cs.HC}
}