Towards Accurate and Efficient Document Analytics with Large Language Models
<|reference_start|>Towards Accurate and Efficient Document Analytics with Large Language Models: Unstructured data formats account for over 80% of the data currently stored, and extracting value from such formats remains a considerable challenge. In particular, current approaches for managing unstructured documents do not support ad-hoc analytical queries on document collections. Moreover, Large Language Models (LLMs) directly applied to the documents themselves, or on portions of documents through a process of Retrieval-Augmented Generation (RAG), fail to provide high accuracy query results, and in the LLM-only case, additionally incur high costs. Since many unstructured documents in a collection often follow similar templates that impart a common semantic structure, we introduce ZenDB, a document analytics system that leverages this semantic structure, coupled with LLMs, to answer ad-hoc SQL queries on document collections. ZenDB efficiently extracts semantic hierarchical structures from such templatized documents, and introduces a novel query engine that leverages these structures for accurate and cost-effective query execution. Users can impose a schema on their documents, and query it, all via SQL. Extensive experiments on three real-world document collections demonstrate ZenDB's benefits, achieving up to 30% cost savings compared to LLM-based baselines, while maintaining or improving accuracy, and surpassing RAG-based baselines by up to 61% in precision and 80% in recall, at a marginally higher cost.<|reference_end|>
@article{lin2024towards,
    title={Towards Accurate and Efficient Document Analytics with Large Language
  Models},
    author={Yiming Lin, Madelon Hulsebos, Ruiying Ma, Shreya Shankar, Sepanta
  Zeigham, Aditya G. Parameswaran, Eugene Wu},
    journal={arXiv preprint arXiv:2405.04674},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.04674},
    primaryClass={cs.DB}
}