Papers
arxiv:2507.15061

WebShaper: Agentically Data Synthesizing via Information-Seeking Formalization

Published on Jul 20
· Submitted by callanwu on Jul 22
Authors:
,
,
,
,
,
,

Abstract

A formalization-driven framework called WebShaper synthesizes information-seeking datasets using set theory and Knowledge Projections, enhancing the performance of LLM-powered agents on open-ended tasks.

AI-generated summary

The advent of Large Language Model (LLM)-powered agents has revolutionized artificial intelligence by enabling solutions to complex, open-ended tasks through web-based information-seeking (IS) capabilities. The scarcity of high-quality training data has limited the development of IS agents. Existing approaches typically adopt an information-driven paradigm that first collects web data and then generates questions based on the retrieval. However, this may lead to inconsistency between information structure and reasoning structure, question and answer. To mitigate, we propose a formalization-driven IS data synthesis framework WebShaper to construct a dataset. WebShaper systematically formalizes IS tasks through set theory. Central to the formalization is the concept of Knowledge Projections (KP), which enables precise control over reasoning structure by KP operation compositions. During synthesis, we begin by creating seed tasks, then use a multi-step expansion process. At each step, an agentic Expander expands the current formal question more complex with retrieval and validation tools based on our formalization. We train our model on the synthesized dataset. Experiment results demonstrate that WebShaper achieves state-of-the-art performance among open-sourced IS agents on GAIA and WebWalkerQA benchmarks.

Community

Paper author Paper submitter

The advent of Large Language Model (LLM)-powered agents has revolutionized artificial intelligence by enabling solutions to complex, open-ended tasks through web-based information-seeking (IS) capabilities. The scarcity of high-quality training data has limited the development of IS agents. Existing approaches typically adopt an information-driven paradigm that first collects web data and then generates questions based on the retrieval. However, this may lead to inconsistency between information structure and reasoning structure, question and answer. To mitigate, we propose a formalization-driven IS data synthesis framework, WebShaper, to construct a dataset. WebShaper systematically formalizes IS tasks through set theory. Central to the formalization is the concept of Knowledge Projections (KP), which enables precise control over reasoning structure by KP operation compositions. During synthesis, we begin by creating seed tasks, then use a multi-step expansion process. At each step, an agentic Expander expands the current formal question more complex with retrieval and validation tools based on our formalization. We train our model on the synthesized dataset. Experiment results demonstrate that WebShaper achieves state-of-the-art performance among open-sourced IS agents on GAIA and WebWalkerQA benchmarks.

·

You guys are so fast!!!

Paper author Paper submitter

We achieve new state-of-the-art results on GAIA (60.19) and WebWalkerQA (52.50) benchmarks.

截屏2025-07-22 11.08.36.png

WebShaper is a dataset for training information-seeking agents. We release 500 question-answer pairs in HuggingFace and ModelScope.

Paper author Paper submitter

More details are presented in https://github.com/Alibaba-NLP/WebAgent.

@callanwu Congrats on this great work. Have you tested the model on BrowseCamp? What's the performance there.

Sign up or log in to comment

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2507.15061 in a model README.md to link it from this page.

Datasets citing this paper 1

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2507.15061 in a Space README.md to link it from this page.

Collections including this paper 4