title: LLM-Powered GUI Agents in Phone Automation: Surveying Progress and
    Prospects
summary: With the rapid rise of large language models (LLMs), phone automation has
  undergone transformative changes. This paper systematically reviews LLM-driven
  phone GUI agents, highlighting their evolution from script-based automation to
  intelligent, adaptive systems. We first contextualize key challenges, (i)
  limited generality, (ii) high maintenance overhead, and (iii) weak intent
  comprehension, and show how LLMs address these issues through advanced language
  understanding, multimodal perception, and robust decision-making. We then
  propose a taxonomy covering fundamental agent frameworks (single-agent,
  multi-agent, plan-then-act), modeling approaches (prompt engineering,
  training-based), and essential datasets and benchmarks. Furthermore, we detail
  task-specific architectures, supervised fine-tuning, and reinforcement learning
  strategies that bridge user intent and GUI operations. Finally, we discuss open
  challenges such as dataset diversity, on-device deployment efficiency,
  user-centric adaptation, and security concerns, offering forward-looking
  insights into this rapidly evolving field. By providing a structured overview
  and identifying pressing research gaps, this paper serves as a definitive
  reference for researchers and practitioners seeking to harness LLMs in
  designing scalable, user-friendly phone GUI agents.
paper_authors._id: 6810317e007d579cbf5200ba
paper_authors.name: Guangyi Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-29T12:15:01.850Z
paper_authors._id: 6108ae87823007eaf0c7bd1e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6108ae87823007eaf0c7bd1e/dKjdx9I5waJs6oUQ0_mmT.png
paper_authors.fullname: Guangyi Liu
paper_authors.type: user
paper_authors.user: guangyil
paper_authors._id: 6810317e007d579cbf5200bb
paper_authors.name: Pengxiang Zhao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-29T07:59:51.686Z
paper_authors._id: 65a088f4300957620ba45c70
paper_authors.avatarUrl: /avatars/56ed45e10d3455531979f30881b2d3f9.svg
paper_authors.fullname: pengxiang zhao
paper_authors.type: user
paper_authors.user: Pengxiangzhao
paper_authors._id: 6810317e007d579cbf5200bc
paper_authors.name: Liang Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-29T08:01:16.707Z
paper_authors._id: 667b91162bfe908436900faa
paper_authors.avatarUrl: /avatars/daeaf058ec1df5307996895a5cbba052.svg
paper_authors.fullname: Liang Liu
paper_authors.type: user
paper_authors.user: melpancake
paper_authors._id: 6810317e007d579cbf5200bd
paper_authors.name: Yaxuan Guo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-29T12:15:15.455Z
paper_authors._id: 65c237220c57a7141888363e
paper_authors.avatarUrl: /avatars/ce43c52f47d524c5b747523058946325.svg
paper_authors.fullname: guoyaxuan
paper_authors.type: user
paper_authors.user: guoyaxuan0106
paper_authors._id: 6810317e007d579cbf5200be
paper_authors.name: Han Xiao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T09:01:47.021Z
paper_authors._id: 666aa99cd1652853e4f9a8b9
paper_authors.avatarUrl: /avatars/7cd5a0c34b5ccb8eff5a353d88d15a93.svg
paper_authors.fullname: HanXiao
paper_authors.type: user
paper_authors.user: HanXiao1999
paper_authors._id: 6810317e007d579cbf5200bf
paper_authors.name: Weifeng Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-29T12:15:29.782Z
paper_authors._id: 66026c9068d519ed32519e9c
paper_authors.avatarUrl: /avatars/8fa051312c713772e5b8ba65989ff7f5.svg
paper_authors.fullname: Weifeng Lin
paper_authors.type: user
paper_authors.user: Afeng-x
paper_authors._id: 6810317e007d579cbf5200c0
paper_authors.name: Yuxiang Chai
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-29T08:01:19.091Z
paper_authors._id: 6458ce236fa580137af5aa95
paper_authors.avatarUrl: /avatars/db65a7332e375eb5daad5c1b076b1e3b.svg
paper_authors.fullname: Yuxiang Chai
paper_authors.type: user
paper_authors.user: Yuxiang007
paper_authors._id: 6810317e007d579cbf5200c1
paper_authors.name: Yue Han
paper_authors._id: 6810317e007d579cbf5200c2
paper_authors.name: Shuai Ren
paper_authors._id: 6810317e007d579cbf5200c3
paper_authors.name: Hao Wang
paper_authors._id: 6810317e007d579cbf5200c4
paper_authors.name: Xiaoyu Liang
paper_authors._id: 6810317e007d579cbf5200c5
paper_authors.name: Wenhao Wang
paper_authors._id: 6810317e007d579cbf5200c6
paper_authors.name: Tianze Wu
paper_authors._id: 6810317e007d579cbf5200c7
paper_authors.name: Linghao Li
paper_authors._id: 6810317e007d579cbf5200c8
paper_authors.name: Hao Wang
paper_authors._id: 6810317e007d579cbf5200c9
paper_authors.name: Guanjing Xiong
paper_authors._id: 6810317e007d579cbf5200ca
paper_authors.name: Yong Liu
paper_authors._id: 6810317e007d579cbf5200cb
paper_authors.name: Hongsheng Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-29T12:17:24.303Z
paper_authors._id: 65c04e9c27a5fdca81abcbd9
paper_authors.avatarUrl: /avatars/12a155683c824fa23da4a9e2bed4f64e.svg
paper_authors.fullname: Hongsheng LI
paper_authors.type: user
paper_authors.user: hsli-cuhk
paper_title: LLM-Powered GUI Agents in Phone Automation: Surveying Progress and
    Prospects
paper_summary: With the rapid rise of large language models (LLMs), phone automation has
  undergone transformative changes. This paper systematically reviews LLM-driven
  phone GUI agents, highlighting their evolution from script-based automation to
  intelligent, adaptive systems. We first contextualize key challenges, (i)
  limited generality, (ii) high maintenance overhead, and (iii) weak intent
  comprehension, and show how LLMs address these issues through advanced language
  understanding, multimodal perception, and robust decision-making. We then
  propose a taxonomy covering fundamental agent frameworks (single-agent,
  multi-agent, plan-then-act), modeling approaches (prompt engineering,
  training-based), and essential datasets and benchmarks. Furthermore, we detail
  task-specific architectures, supervised fine-tuning, and reinforcement learning
  strategies that bridge user intent and GUI operations. Finally, we discuss open
  challenges such as dataset diversity, on-device deployment efficiency,
  user-centric adaptation, and security concerns, offering forward-looking
  insights into this rapidly evolving field. By providing a structured overview
  and identifying pressing research gaps, this paper serves as a definitive
  reference for researchers and practitioners seeking to harness LLMs in
  designing scalable, user-friendly phone GUI agents.
paper_ai_summary: LLM-driven phone GUI agents evolve from script-based automation to intelligent systems, using advanced language understanding, multimodal perception, and robust decision-making to address generality, maintenance overhead, and intent comprehension challenges.
paper_ai_keywords: LLMs phone GUI agents script-based automation intelligent systems advanced language understanding multimodal perception robust decision-making single-agent multi-agent plan-then-act prompt engineering training-based supervised fine-tuning reinforcement learning user intent GUI operations dataset diversity on-device deployment efficiency user-centric adaptation security concerns