PaSa: An LLM Agent for Comprehensive Academic Paper Search
We introduce PaSa, an advanced Paper Search agent powered by large language
models. PaSa can autonomously make a series of decisions, including invoking
search tools, reading papers, and selecting relevant references, to ultimately
obtain comprehensive and accurate results for complex scholarly queries. We
optimize PaSa using reinforcement learning with a synthetic dataset,
AutoScholarQuery, which includes 35k fine-grained academic queries and
corresponding papers sourced from top-tier AI conference publications.
Additionally, we develop RealScholarQuery, a benchmark collecting real-world
academic queries to assess PaSa performance in more realistic scenarios.
Despite being trained on synthetic data, PaSa significantly outperforms
existing baselines on RealScholarQuery, including Google, Google Scholar,
Google with GPT-4 for paraphrased queries, chatGPT (search-enabled GPT-4o),
GPT-o1, and PaSa-GPT-4o (PaSa implemented by prompting GPT-4o). Notably,
PaSa-7B surpasses the best Google-based baseline, Google with GPT-4o, by 37.78%
in recall@20 and 39.90% in recall@50. It also exceeds PaSa-GPT-4o by 30.36% in
recall and 4.25% in precision. Model, datasets, and code are available at
https://github.com/bytedance/pasa.
678dc283d4a7a158a8e5cf08
Yichen He
admin_assigned
2025-01-20T13:19:13.591Z
60ea81771cc8dc259c58e905
https://cdn-avatars.huggingface.co/v1/production/uploads/60ea81771cc8dc259c58e905/kmGlaNvdS4EEHc_5qongT.jpeg
yichen he
user
hyc2026
678dc283d4a7a158a8e5cf09
Guanhua Huang
678dc283d4a7a158a8e5cf0a
Peiyuan Feng
admin_assigned
2025-01-20T13:19:41.033Z
662608797670fa2bc0d0fa0a
/avatars/eef336e6ba43fae56c90e17f60606f4d.svg
fengpeiyuan
user
fpybytedance
678dc283d4a7a158a8e5cf0b
Yuan Lin
678dc283d4a7a158a8e5cf0c
Yuchen Zhang
678dc283d4a7a158a8e5cf0d
Hang Li
678dc283d4a7a158a8e5cf0e
Weinan E
PaSa: An LLM Agent for Comprehensive Academic Paper Search
We introduce PaSa, an advanced Paper Search agent powered by large language
models. PaSa can autonomously make a series of decisions, including invoking
search tools, reading papers, and selecting relevant references, to ultimately
obtain comprehensive and accurate results for complex scholarly queries. We
optimize PaSa using reinforcement learning with a synthetic dataset,
AutoScholarQuery, which includes 35k fine-grained academic queries and
corresponding papers sourced from top-tier AI conference publications.
Additionally, we develop RealScholarQuery, a benchmark collecting real-world
academic queries to assess PaSa performance in more realistic scenarios.
Despite being trained on synthetic data, PaSa significantly outperforms
existing baselines on RealScholarQuery, including Google, Google Scholar,
Google with GPT-4 for paraphrased queries, chatGPT (search-enabled GPT-4o),
GPT-o1, and PaSa-GPT-4o (PaSa implemented by prompting GPT-4o). Notably,
PaSa-7B surpasses the best Google-based baseline, Google with GPT-4o, by 37.78%
in recall@20 and 39.90% in recall@50. It also exceeds PaSa-GPT-4o by 30.36% in
recall and 4.25% in precision. Model, datasets, and code are available at
https://github.com/bytedance/pasa.
A quality Paper Search agent, PaSa, uses reinforcement learning on synthetic and real academic queries, outperforming Google and GPT-4-based systems.
large language models reinforcement learning synthetic dataset AutoScholarQuery RealScholarQuery recall@20 recall@50 precision