Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interaction
The current paradigm of test-time scaling relies on generating long reasoning
traces ("thinking" more) before producing a response. In agent problems that
require interaction, this can be done by generating thinking traces before
acting in the world. However, this process does not allow agents to acquire new
information from the environment or adapt their behavior over time. In this
work, we propose to scale test-time interaction, an untapped dimension of
test-time scaling that increases the agent's interaction horizon to enable
running rich behaviors such as exploration, backtracking, and dynamic
re-planning within a single rollout. To demonstrate the promise of this scaling
dimension, we study the domain of web agents. We first show that even
prompting-based interaction scaling without any training can improve task
success on web benchmarks non-trivially. Building on this, we introduce TTI
(Test-Time Interaction), a curriculum-based online reinforcement learning (RL)
approach that trains agents by adaptively adjusting their rollout lengths.
Using a Gemma 3 12B model, TTI produces state-of-the-art open-source, open-data
web agents on WebVoyager and WebArena benchmarks. We further show that TTI
enables agents to balance exploration and exploitation adaptively. Our results
establish interaction scaling as a powerful, complementary axis to scaling
per-step compute, offering new avenues for training adaptive agents.
68487bd642e4f9106973f16d
Junhong Shen
68487bd642e4f9106973f16e
Hao Bai
claimed_verified
2025-06-11T10:08:42.049Z
62927c2e56fedc76e396b3ca
https://cdn-avatars.huggingface.co/v1/production/uploads/1678105603200-62927c2e56fedc76e396b3ca.jpeg
HAO BAI
user
JackBAI
68487bd642e4f9106973f16f
Lunjun Zhang
68487bd642e4f9106973f170
Yifei Zhou
68487bd642e4f9106973f171
Amrith Setlur
68487bd642e4f9106973f172
Shengbang Tong
68487bd642e4f9106973f173
Diego Caples
68487bd642e4f9106973f174
Nan Jiang
68487bd642e4f9106973f175
Tong Zhang
68487bd642e4f9106973f176
Ameet Talwalkar
68487bd642e4f9106973f177
Aviral Kumar
Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interaction
The current paradigm of test-time scaling relies on generating long reasoning
traces ("thinking" more) before producing a response. In agent problems that
require interaction, this can be done by generating thinking traces before
acting in the world. However, this process does not allow agents to acquire new
information from the environment or adapt their behavior over time. In this
work, we propose to scale test-time interaction, an untapped dimension of
test-time scaling that increases the agent's interaction horizon to enable
running rich behaviors such as exploration, backtracking, and dynamic
re-planning within a single rollout. To demonstrate the promise of this scaling
dimension, we study the domain of web agents. We first show that even
prompting-based interaction scaling without any training can improve task
success on web benchmarks non-trivially. Building on this, we introduce TTI
(Test-Time Interaction), a curriculum-based online reinforcement learning (RL)
approach that trains agents by adaptively adjusting their rollout lengths.
Using a Gemma 3 12B model, TTI produces state-of-the-art open-source, open-data
web agents on WebVoyager and WebArena benchmarks. We further show that TTI
enables agents to balance exploration and exploitation adaptively. Our results
establish interaction scaling as a powerful, complementary axis to scaling
per-step compute, offering new avenues for training adaptive agents.
Test-Time Interaction (TTI) improves web agent performance by scaling interaction, enabling adaptive behavior and balancing exploration and exploitation without adding per-step compute.
test-time scaling thinking traces agent interaction interaction horizon exploration backtracking dynamic re-planning rollout curriculum-based online reinforcement learning (RL) Gemma 3 12B model WebVoyager WebArena adaptive agents