title: TreeHop: Generate and Filter Next Query Embeddings Efficiently for
    Multi-hop Question Answering
summary: Retrieval-augmented generation (RAG) systems face significant challenges in
  multi-hop question answering (MHQA), where complex queries require synthesizing
  information across multiple document chunks. Existing approaches typically rely
  on iterative LLM-based query rewriting and routing, resulting in high
  computational costs due to repeated LLM invocations and multi-stage processes.
  To address these limitations, we propose TreeHop, an embedding-level framework
  without the need for LLMs in query refinement. TreeHop dynamically updates
  query embeddings by fusing semantic information from prior queries and
  retrieved documents, enabling iterative retrieval through embedding-space
  operations alone. This method replaces the traditional
  "Retrieve-Rewrite-Vectorize-Retrieve" cycle with a streamlined
  "Retrieve-Embed-Retrieve" loop, significantly reducing computational overhead.
  Moreover, a rule-based stop criterion is introduced to further prune redundant
  retrievals, balancing efficiency and recall rate. Experimental results show
  that TreeHop rivals advanced RAG methods across three open-domain MHQA
  datasets, achieving comparable performance with only 5\%-0.4\% of the model
  parameter size and reducing the query latency by approximately 99\% compared to
  concurrent approaches. This makes TreeHop a faster and more cost-effective
  solution for deployment in a range of knowledge-intensive applications. For
  reproducibility purposes, codes and data are available here:
  https://github.com/allen-li1231/TreeHop.
paper_authors._id: 6811ee5994d2a4531ca2b697
paper_authors.name: Zhonghao Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-30T12:24:10.370Z
paper_authors._id: 66a378a7a24a4e3865e6aadf
paper_authors.avatarUrl: /avatars/ff1560941fad2a6224418f399d38b1cc.svg
paper_authors.fullname: Allen Li
paper_authors.type: user
paper_authors.user: allen-li1231
paper_authors._id: 6811ee5994d2a4531ca2b698
paper_authors.name: Kunpeng Zhang
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-04-30T09:33:14.657Z
paper_authors._id: 65cb93826427380bc218c408
paper_authors.avatarUrl: /avatars/73d834cf4ce7d09d6adb68b1e974cbf8.svg
paper_authors.fullname: KZ Zhang
paper_authors.type: user
paper_authors.user: kpzhang1028
paper_authors._id: 6811ee5994d2a4531ca2b699
paper_authors.name: Jinghuai Ou
paper_authors._id: 6811ee5994d2a4531ca2b69a
paper_authors.name: Shuliang Liu
paper_authors._id: 6811ee5994d2a4531ca2b69b
paper_authors.name: Xuming Hu
paper_title: TreeHop: Generate and Filter Next Query Embeddings Efficiently for
    Multi-hop Question Answering
paper_summary: Retrieval-augmented generation (RAG) systems face significant challenges in
  multi-hop question answering (MHQA), where complex queries require synthesizing
  information across multiple document chunks. Existing approaches typically rely
  on iterative LLM-based query rewriting and routing, resulting in high
  computational costs due to repeated LLM invocations and multi-stage processes.
  To address these limitations, we propose TreeHop, an embedding-level framework
  without the need for LLMs in query refinement. TreeHop dynamically updates
  query embeddings by fusing semantic information from prior queries and
  retrieved documents, enabling iterative retrieval through embedding-space
  operations alone. This method replaces the traditional
  "Retrieve-Rewrite-Vectorize-Retrieve" cycle with a streamlined
  "Retrieve-Embed-Retrieve" loop, significantly reducing computational overhead.
  Moreover, a rule-based stop criterion is introduced to further prune redundant
  retrievals, balancing efficiency and recall rate. Experimental results show
  that TreeHop rivals advanced RAG methods across three open-domain MHQA
  datasets, achieving comparable performance with only 5\%-0.4\% of the model
  parameter size and reducing the query latency by approximately 99\% compared to
  concurrent approaches. This makes TreeHop a faster and more cost-effective
  solution for deployment in a range of knowledge-intensive applications. For
  reproducibility purposes, codes and data are available here:
  https://github.com/allen-li1231/TreeHop.
paper_ai_summary: TreeHop, an embedding-level framework for multi-hop question answering, significantly reduces computational costs compared to traditional retrieval-augmented generation systems by eliminating the need for iterative LLM-based query refinement.
paper_ai_keywords: retrieval-augmented generation multi-hop question answering query rewriting LLMs TreeHop embedding-level framework query embeddings embedding-space operations rule-based stop criterion open-domain MHQA datasets