SPAGHETTI: Open-Domain Question Answering from Heterogeneous Data Sources with Retrieval and Semantic Parsing
<|reference_start|>SPAGHETTI: Open-Domain Question Answering from Heterogeneous Data Sources with Retrieval and Semantic Parsing: We introduce SPAGHETTI: Semantic Parsing Augmented Generation for Hybrid English information from Text Tables and Infoboxes, a hybrid question-answering (QA) pipeline that utilizes information from heterogeneous knowledge sources, including knowledge base, text, tables, and infoboxes. Our LLM-augmented approach achieves state-of-the-art performance on the Compmix dataset, the most comprehensive heterogeneous open-domain QA dataset, with 56.5% exact match (EM) rate. More importantly, manual analysis on a sample of the dataset suggests that SPAGHETTI is more than 90% accurate, indicating that EM is no longer suitable for assessing the capabilities of QA systems today.<|reference_end|>
@article{zhang2024spaghetti:,
    title={SPAGHETTI: Open-Domain Question Answering from Heterogeneous Data
  Sources with Retrieval and Semantic Parsing},
    author={Heidi C. Zhang, Sina J. Semnani, Farhad Ghassemi, Jialiang Xu,
  Shicheng Liu, Monica S. Lam},
    journal={arXiv preprint arXiv:2406.00562},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.00562},
    primaryClass={cs.CL}
}