Ingest-And-Ground: Dispelling Hallucinations from Continually-Pretrained LLMs with RAG
<|reference_start|>Ingest-And-Ground: Dispelling Hallucinations from Continually-Pretrained LLMs with RAG: This paper presents new methods that have the potential to improve privacy process efficiency with LLM and RAG. To reduce hallucination, we continually pre-train the base LLM model with a privacy-specific knowledge base and then augment it with a semantic RAG layer. Our evaluations demonstrate that this approach enhances the model performance (as much as doubled metrics compared to out-of-box LLM) in handling privacy-related queries, by grounding responses with factual information which reduces inaccuracies.<|reference_end|>
@article{fang2024ingest-and-ground:,
    title={Ingest-And-Ground: Dispelling Hallucinations from Continually-Pretrained
  LLMs with RAG},
    author={Chenhao Fang, Derek Larson, Shitong Zhu, Sophie Zeng, Wendy Summer,
  Yanqing Peng, Yuriy Hulovatyy, Rajeev Rao, Gabriel Forgues, Arya Pudota, Alex
  Goncalves, Herv'e Robert},
    journal={arXiv preprint arXiv:2410.02825},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.02825},
    primaryClass={cs.CL cs.CR}
}