PEER: Expertizing Domain-Specific Tasks with a Multi-Agent Framework and Tuning Methods
<|reference_start|>PEER: Expertizing Domain-Specific Tasks with a Multi-Agent Framework and Tuning Methods: In domain-specific applications, GPT-4, augmented with precise prompts or Retrieval-Augmented Generation (RAG), shows notable potential but faces the critical tri-lemma of performance, cost, and data privacy. High performance requires sophisticated processing techniques, yet managing multiple agents within a complex workflow often proves costly and challenging. To address this, we introduce the PEER (Plan, Execute, Express, Review) multi-agent framework. This systematizes domain-specific tasks by integrating precise question decomposition, advanced information retrieval, comprehensive summarization, and rigorous self-assessment. Given the concerns of cost and data privacy, enterprises are shifting from proprietary models like GPT-4 to custom models, striking a balance between cost, security, and performance. We developed industrial practices leveraging online data and user feedback for efficient model tuning. This study provides best practice guidelines for applying multi-agent systems in domain-specific problem-solving and implementing effective agent tuning strategies. Our empirical studies, particularly in the financial question-answering domain, demonstrate that our approach achieves 95.0% of GPT-4's performance, while effectively managing costs and ensuring data privacy.<|reference_end|>
@article{wang2024peer:,
    title={PEER: Expertizing Domain-Specific Tasks with a Multi-Agent Framework and
  Tuning Methods},
    author={Yiying Wang, Xiaojing Li, Binzhu Wang, Yueyang Zhou, Yingru Lin, Han
  Ji, Hong Chen, Jinshi Zhang, Fei Yu, Zewei Zhao, Song Jin, Renji Gong,
  Wanqing Xu},
    journal={arXiv preprint arXiv:2407.06985},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.06985},
    primaryClass={cs.AI}
}