LOTUS: Enabling Semantic Queries with LLMs Over Tables of Unstructured and Structured Data
<|reference_start|>LOTUS: Enabling Semantic Queries with LLMs Over Tables of Unstructured and Structured Data: The semantic capabilities of language models (LMs) have the potential to enable rich analytics and reasoning over vast knowledge corpora. Unfortunately, existing systems lack high-level abstractions to perform semantic queries at scale. We introduce semantic operators, a declarative programming interface that extends the relational model with composable AI-based operations for semantic queries over datasets (e.g., sorting or aggregating records using natural language criteria). Each operator can be implemented and optimized in multiple ways, opening a rich space for execution plans similar to relational operators. We implement our operators and several optimizations for them in LOTUS, an open-source query engine with a Pandas-like API.   We demonstrate LOTUS' effectiveness across a series of real applications, including fact-checking, extreme multi-label classification, and search. We find that LOTUS' programming model is highly expressive, capturing state-of-the-art query pipelines with low development overhead. Specifically, on the FEVER dataset, LOTUS' programs can reproduce FacTool, a recent state-of-the-art fact-checking pipeline, in few lines of code, and implement a new pipeline that improves accuracy by $9.5\%$, while offering $7-34\times$ lower execution time. In the extreme multi-label classification task on the BioDEX dataset, LOTUS reproduces state-of-the art result quality with its join operator, while providing an efficient algorithm that runs $800\times$ faster than a naive join. In the search and ranking application, LOTUS allows a simple composition of operators to achieve $5.9 - 49.4\%$ higher nDCG@10 than the vanilla retriever and re-ranker, while also providing query efficiency, with $1.67 - 10\times$ lower execution time than LM-based ranking methods used by prior works. LOTUS is publicly available at https://github.com/stanford-futuredata/lotus.<|reference_end|>
@article{patel2024lotus:,
    title={LOTUS: Enabling Semantic Queries with LLMs Over Tables of Unstructured
  and Structured Data},
    author={Liana Patel, Siddharth Jha, Carlos Guestrin, Matei Zaharia},
    journal={arXiv preprint arXiv:2407.11418},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.11418},
    primaryClass={cs.DB cs.AI cs.CL}
}