title: Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Tasks
summary: Smartphones have become indispensable in modern life, yet navigating complex
  tasks on mobile devices often remains frustrating. Recent advancements in large
  multimodal model (LMM)-based mobile agents have demonstrated the ability to
  perceive and act in mobile environments. However, current approaches face
  significant limitations: they fall short in addressing real-world human needs,
  struggle with reasoning-intensive and long-horizon tasks, and lack mechanisms
  to learn and improve from prior experiences. To overcome these challenges, we
  introduce Mobile-Agent-E, a hierarchical multi-agent framework capable of
  self-evolution through past experience. By hierarchical, we mean an explicit
  separation of high-level planning and low-level action execution. The framework
  comprises a Manager, responsible for devising overall plans by breaking down
  complex tasks into subgoals, and four subordinate agents--Perceptor, Operator,
  Action Reflector, and Notetaker--which handle fine-grained visual perception,
  immediate action execution, error verification, and information aggregation,
  respectively. Mobile-Agent-E also features a novel self-evolution module which
  maintains a persistent long-term memory comprising Tips and Shortcuts. Tips are
  general guidance and lessons learned from prior tasks on how to effectively
  interact with the environment. Shortcuts are reusable, executable sequences of
  atomic operations tailored for specific subroutines. The inclusion of Tips and
  Shortcuts facilitates continuous refinement in performance and efficiency.
  Alongside this framework, we introduce Mobile-Eval-E, a new benchmark featuring
  complex mobile tasks requiring long-horizon, multi-app interactions. Empirical
  results show that Mobile-Agent-E achieves a 22% absolute improvement over
  previous state-of-the-art approaches across three foundation model backbones.
  Project page: https://x-plug.github.io/MobileAgent.
paper_authors._id: 6790791b203b95acf96ebf45
paper_authors.name: Zhenhailong Wang
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-01-22T04:50:40.468Z
paper_authors._id: 628d7265db4cd1d1717c884f
paper_authors.avatarUrl: /avatars/dff2a3dd10d84b4a73fa486402de7219.svg
paper_authors.fullname: Zhenhailong Wang
paper_authors.type: user
paper_authors.user: mikewang
paper_authors._id: 6790791b203b95acf96ebf46
paper_authors.name: Haiyang Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-22T10:08:13.360Z
paper_authors._id: 645b10e80c73ea27d13f7aca
paper_authors.avatarUrl: /avatars/95e565306472a15067440b5b43e07a6f.svg
paper_authors.fullname: xuhaiyang
paper_authors.type: user
paper_authors.user: xhyandwyy
paper_authors._id: 6790791b203b95acf96ebf47
paper_authors.name: Junyang Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-22T15:47:37.467Z
paper_authors._id: 6438f6415aa69077ffb16942
paper_authors.avatarUrl: /avatars/c83dbd3e10e88db97c2a86092bad5917.svg
paper_authors.fullname: Junyang Wang
paper_authors.type: user
paper_authors.user: junyangwang0410
paper_authors._id: 6790791b203b95acf96ebf48
paper_authors.name: Xi Zhang
paper_authors._id: 6790791b203b95acf96ebf49
paper_authors.name: Ming Yan
paper_authors._id: 6790791b203b95acf96ebf4a
paper_authors.name: Ji Zhang
paper_authors._id: 6790791b203b95acf96ebf4b
paper_authors.name: Fei Huang
paper_authors._id: 6790791b203b95acf96ebf4c
paper_authors.name: Heng Ji
paper_title: Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Tasks
paper_summary: Smartphones have become indispensable in modern life, yet navigating complex
  tasks on mobile devices often remains frustrating. Recent advancements in large
  multimodal model (LMM)-based mobile agents have demonstrated the ability to
  perceive and act in mobile environments. However, current approaches face
  significant limitations: they fall short in addressing real-world human needs,
  struggle with reasoning-intensive and long-horizon tasks, and lack mechanisms
  to learn and improve from prior experiences. To overcome these challenges, we
  introduce Mobile-Agent-E, a hierarchical multi-agent framework capable of
  self-evolution through past experience. By hierarchical, we mean an explicit
  separation of high-level planning and low-level action execution. The framework
  comprises a Manager, responsible for devising overall plans by breaking down
  complex tasks into subgoals, and four subordinate agents--Perceptor, Operator,
  Action Reflector, and Notetaker--which handle fine-grained visual perception,
  immediate action execution, error verification, and information aggregation,
  respectively. Mobile-Agent-E also features a novel self-evolution module which
  maintains a persistent long-term memory comprising Tips and Shortcuts. Tips are
  general guidance and lessons learned from prior tasks on how to effectively
  interact with the environment. Shortcuts are reusable, executable sequences of
  atomic operations tailored for specific subroutines. The inclusion of Tips and
  Shortcuts facilitates continuous refinement in performance and efficiency.
  Alongside this framework, we introduce Mobile-Eval-E, a new benchmark featuring
  complex mobile tasks requiring long-horizon, multi-app interactions. Empirical
  results show that Mobile-Agent-E achieves a 22% absolute improvement over
  previous state-of-the-art approaches across three foundation model backbones.
  Project page: https://x-plug.github.io/MobileAgent.
paper_ai_summary: A hierarchical multi-agent framework called Mobile-Agent-E enhances mobile task navigation through self-evolution, improving performance and efficiency on complex long-horizon tasks.
paper_ai_keywords: large multimodal model (LMM) mobile agents hierarchical multi-agent framework self-evolution Manager Perceptor Operator Action Reflector Notetaker long-term memory Tips Shortcuts Mobile-Eval-E long-horizon multi-app interactions