InspectorRAGet: An Introspection Platform for RAG Evaluation
<|reference_start|>InspectorRAGet: An Introspection Platform for RAG Evaluation: Large Language Models (LLM) have become a popular approach for implementing Retrieval Augmented Generation (RAG) systems, and a significant amount of effort has been spent on building good models and metrics. In spite of increased recognition of the need for rigorous evaluation of RAG systems, few tools exist that go beyond the creation of model output and automatic calculation. We present InspectorRAGet, an introspection platform for RAG evaluation. InspectorRAGet allows the user to analyze aggregate and instance-level performance of RAG systems, using both human and algorithmic metrics as well as annotator quality. InspectorRAGet is suitable for multiple use cases and is available publicly to the community. The demo video is available at https://youtu.be/MJhe8QIXcEc<|reference_end|>
@article{fadnis2024inspectorraget:,
    title={InspectorRAGet: An Introspection Platform for RAG Evaluation},
    author={Kshitij Fadnis, Siva Sankalp Patel, Odellia Boni, Yannis Katsis, Sara
  Rosenthal, Benjamin Sznajder and Marina Danilevsky},
    journal={arXiv preprint arXiv:2404.17347},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.17347},
    primaryClass={cs.SE cs.HC}
}