Mobile-Agent-v2: Mobile Device Operation Assistant with Effective
  Navigation via Multi-Agent Collaboration
Mobile device operation tasks are increasingly becoming a popular multi-modal
AI application scenario. Current Multi-modal Large Language Models (MLLMs),
constrained by their training data, lack the capability to function effectively
as operation assistants. Instead, MLLM-based agents, which enhance capabilities
through tool invocation, are gradually being applied to this scenario. However,
the two major navigation challenges in mobile device operation tasks, task
progress navigation and focus content navigation, are significantly complicated
under the single-agent architecture of existing work. This is due to the overly
long token sequences and the interleaved text-image data format, which limit
performance. To address these navigation challenges effectively, we propose
Mobile-Agent-v2, a multi-agent architecture for mobile device operation
assistance. The architecture comprises three agents: planning agent, decision
agent, and reflection agent. The planning agent generates task progress, making
the navigation of history operations more efficient. To retain focus content,
we design a memory unit that updates with task progress. Additionally, to
correct erroneous operations, the reflection agent observes the outcomes of
each operation and handles any mistakes accordingly. Experimental results
indicate that Mobile-Agent-v2 achieves over a 30% improvement in task
completion compared to the single-agent architecture of Mobile-Agent. The code
is open-sourced at https://github.com/X-PLUG/MobileAgent.
666113b372f344f198df601c
Junyang Wang
admin_assigned
2024-06-06T07:16:21.069Z
6438f6415aa69077ffb16942
/avatars/c83dbd3e10e88db97c2a86092bad5917.svg
Junyang Wang
user
junyangwang0410
666113b372f344f198df601d
Haiyang Xu
claimed_verified
2024-06-06T06:55:55.255Z
645b10e80c73ea27d13f7aca
/avatars/95e565306472a15067440b5b43e07a6f.svg
xuhaiyang
user
xhyandwyy
666113b372f344f198df601e
Haitao Jia
666113b372f344f198df601f
Xi Zhang
666113b372f344f198df6020
Ming Yan
admin_assigned
2024-06-06T07:14:41.428Z
64771cfdd7cf39f2e9381aa9
/avatars/48adf00c3b653df02628f80511639e19.svg
Ming
user
MingYan123
666113b372f344f198df6021
Weizhou Shen
admin_assigned
2024-06-06T07:14:54.788Z
64777a346e6c7ac608c1e9bf
/avatars/b0e65ba781c90c2560606eb5467101eb.svg
Weizhou Shen
user
shenwzh3
666113b372f344f198df6022
Ji Zhang
666113b372f344f198df6023
Fei Huang
admin_assigned
2024-06-06T07:13:19.531Z
635b8b6a37c6a2c12e2cce00
/avatars/229fb72180529141515d1df797b33709.svg
Fei Huang
user
hzhwcmhf
666113b372f344f198df6024
Jitao Sang
claimed_verified
2024-12-04T09:47:26.638Z
6494457c6339264dd78bcb95
/avatars/d87842251f1a43f50cc827f0e2a995ee.svg
sdzy
user
sdzy
Mobile-Agent-v2: Mobile Device Operation Assistant with Effective
  Navigation via Multi-Agent Collaboration
Mobile device operation tasks are increasingly becoming a popular multi-modal
AI application scenario. Current Multi-modal Large Language Models (MLLMs),
constrained by their training data, lack the capability to function effectively
as operation assistants. Instead, MLLM-based agents, which enhance capabilities
through tool invocation, are gradually being applied to this scenario. However,
the two major navigation challenges in mobile device operation tasks, task
progress navigation and focus content navigation, are significantly complicated
under the single-agent architecture of existing work. This is due to the overly
long token sequences and the interleaved text-image data format, which limit
performance. To address these navigation challenges effectively, we propose
Mobile-Agent-v2, a multi-agent architecture for mobile device operation
assistance. The architecture comprises three agents: planning agent, decision
agent, and reflection agent. The planning agent generates task progress, making
the navigation of history operations more efficient. To retain focus content,
we design a memory unit that updates with task progress. Additionally, to
correct erroneous operations, the reflection agent observes the outcomes of
each operation and handles any mistakes accordingly. Experimental results
indicate that Mobile-Agent-v2 achieves over a 30% improvement in task
completion compared to the single-agent architecture of Mobile-Agent. The code
is open-sourced at https://github.com/X-PLUG/MobileAgent.
Mobile-Agent-v2, a multi-agent system with planning, decision, and reflection components, improves task completion in mobile device operations by addressing navigation challenges and handling errors.
Multi-modal Large Language Models (MLLMs) Mobile-Agent-v2 multi-agent architecture planning agent decision agent reflection agent task progress navigation focus content navigation memory unit erroneous operations