ERATTA: Extreme RAG for Table To Answers with Large Language Models
<|reference_start|>ERATTA: Extreme RAG for Table To Answers with Large Language Models: Large language models (LLMs) with retrieval augmented-generation (RAG) have been the optimal choice for scalable generative AI solutions in the recent past. Although RAG implemented with AI agents (agentic-RAG) has been recently popularized, its suffers from unstable cost and unreliable performances for Enterprise-level data-practices. Most existing use-cases that incorporate RAG with LLMs have been either generic or extremely domain specific, thereby questioning the scalability and generalizability of RAG-LLM approaches. In this work, we propose a unique LLM-based system where multiple LLMs can be invoked to enable data authentication, user-query routing, data-retrieval and custom prompting for question-answering capabilities from Enterprise-data tables. The source tables here are highly fluctuating and large in size and the proposed framework enables structured responses in under 10 seconds per query. Additionally, we propose a five metric scoring module that detects and reports hallucinations in the LLM responses. Our proposed system and scoring metrics achieve >90% confidence scores across hundreds of user queries in the sustainability, financial health and social media domains. Extensions to the proposed extreme RAG architectures can enable heterogeneous source querying using LLMs.<|reference_end|>
@article{roychowdhury2024eratta:,
    title={ERATTA: Extreme RAG for Table To Answers with Large Language Models},
    author={Sohini Roychowdhury, Marko Krema, Anvar Mahammad, Brian Moore, Arijit
  Mukherjee, Punit Prakashchandra},
    journal={arXiv preprint arXiv:2405.03963},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.03963},
    primaryClass={cs.AI cs.LG}
}