title: UI-Genie: A Self-Improving Approach for Iteratively Boosting MLLM-based
    Mobile GUI Agents
summary: In this paper, we introduce UI-Genie, a self-improving framework addressing
  two key challenges in GUI agents: verification of trajectory outcome is
  challenging and high-quality training data are not scalable. These challenges
  are addressed by a reward model and a self-improving pipeline, respectively.
  The reward model, UI-Genie-RM, features an image-text interleaved architecture
  that efficiently pro- cesses historical context and unifies action-level and
  task-level rewards. To sup- port the training of UI-Genie-RM, we develop
  deliberately-designed data genera- tion strategies including rule-based
  verification, controlled trajectory corruption, and hard negative mining. To
  address the second challenge, a self-improvement pipeline progressively expands
  solvable complex GUI tasks by enhancing both the agent and reward models
  through reward-guided exploration and outcome verification in dynamic
  environments. For training the model, we generate UI- Genie-RM-517k and
  UI-Genie-Agent-16k, establishing the first reward-specific dataset for GUI
  agents while demonstrating high-quality synthetic trajectory gen- eration
  without manual annotation. Experimental results show that UI-Genie achieves
  state-of-the-art performance across multiple GUI agent benchmarks with three
  generations of data-model self-improvement. We open-source our complete
  framework implementation and generated datasets to facilitate further research
  in https://github.com/Euphoria16/UI-Genie.
paper_authors._id: 683698f3c32e462c40a9188f
paper_authors.name: Han Xiao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T08:56:45.845Z
paper_authors._id: 666aa99cd1652853e4f9a8b9
paper_authors.avatarUrl: /avatars/7cd5a0c34b5ccb8eff5a353d88d15a93.svg
paper_authors.fullname: HanXiao
paper_authors.type: user
paper_authors.user: HanXiao1999
paper_authors._id: 683698f3c32e462c40a91890
paper_authors.name: Guozhi Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-28T12:17:24.039Z
paper_authors._id: 65446e3fdc39aa9fabba8fd5
paper_authors.avatarUrl: /avatars/41321bd37d007387d1d363b6b4fd2079.svg
paper_authors.fullname: guozhi wang
paper_authors.type: user
paper_authors.user: juice-wang
paper_authors._id: 683698f3c32e462c40a91891
paper_authors.name: Yuxiang Chai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-28T12:17:29.742Z
paper_authors._id: 6458ce236fa580137af5aa95
paper_authors.avatarUrl: /avatars/db65a7332e375eb5daad5c1b076b1e3b.svg
paper_authors.fullname: Yuxiang Chai
paper_authors.type: user
paper_authors.user: Yuxiang007
paper_authors._id: 683698f3c32e462c40a91892
paper_authors.name: Zimu Lu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-28T12:17:35.790Z
paper_authors._id: 64b0bfef2f2f9c345b87e673
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/7O3zCr_IyGvtz0BUdKNKi.jpeg
paper_authors.fullname: Zimu Lu
paper_authors.type: user
paper_authors.user: luzimu
paper_authors._id: 683698f3c32e462c40a91893
paper_authors.name: Weifeng Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-28T12:17:49.451Z
paper_authors._id: 66026c9068d519ed32519e9c
paper_authors.avatarUrl: /avatars/8fa051312c713772e5b8ba65989ff7f5.svg
paper_authors.fullname: Weifeng Lin
paper_authors.type: user
paper_authors.user: Afeng-x
paper_authors._id: 683698f3c32e462c40a91894
paper_authors.name: Hao He
paper_authors._id: 683698f3c32e462c40a91895
paper_authors.name: Lue Fan
paper_authors._id: 683698f3c32e462c40a91896
paper_authors.name: Liuyang Bian
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-28T12:17:55.352Z
paper_authors._id: 66d459ca1dbd780574b42e2d
paper_authors.avatarUrl: /avatars/d573a92487628bf6c727b77eedf80b9d.svg
paper_authors.fullname: bian liuyang
paper_authors.type: user
paper_authors.user: liuyangbian
paper_authors._id: 683698f3c32e462c40a91897
paper_authors.name: Rui Hu
paper_authors._id: 683698f3c32e462c40a91898
paper_authors.name: Liang Liu
paper_authors._id: 683698f3c32e462c40a91899
paper_authors.name: Shuai Ren
paper_authors._id: 683698f3c32e462c40a9189a
paper_authors.name: Yafei Wen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-28T12:18:16.701Z
paper_authors._id: 67134b1ce58b557752c54a1c
paper_authors.avatarUrl: /avatars/5cc07748dac424a6f5c7ca09f1c45068.svg
paper_authors.fullname: yafei wen
paper_authors.type: user
paper_authors.user: wolf1110
paper_authors._id: 683698f3c32e462c40a9189b
paper_authors.name: Xiaoxin Chen
paper_authors._id: 683698f3c32e462c40a9189c
paper_authors.name: Aojun Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T08:56:48.292Z
paper_authors._id: 637de1520d5bb06fbe5207a9
paper_authors.avatarUrl: /avatars/1090851217270c5a858b13e013356d4f.svg
paper_authors.fullname: AJ.Zhou
paper_authors.type: user
paper_authors.user: AJZhou
paper_authors._id: 683698f3c32e462c40a9189d
paper_authors.name: Hongsheng Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-28T12:18:35.337Z
paper_authors._id: 65c04e9c27a5fdca81abcbd9
paper_authors.avatarUrl: /avatars/12a155683c824fa23da4a9e2bed4f64e.svg
paper_authors.fullname: Hongsheng LI
paper_authors.type: user
paper_authors.user: hsli-cuhk
paper_title: UI-Genie: A Self-Improving Approach for Iteratively Boosting MLLM-based
    Mobile GUI Agents
paper_summary: In this paper, we introduce UI-Genie, a self-improving framework addressing
  two key challenges in GUI agents: verification of trajectory outcome is
  challenging and high-quality training data are not scalable. These challenges
  are addressed by a reward model and a self-improving pipeline, respectively.
  The reward model, UI-Genie-RM, features an image-text interleaved architecture
  that efficiently pro- cesses historical context and unifies action-level and
  task-level rewards. To sup- port the training of UI-Genie-RM, we develop
  deliberately-designed data genera- tion strategies including rule-based
  verification, controlled trajectory corruption, and hard negative mining. To
  address the second challenge, a self-improvement pipeline progressively expands
  solvable complex GUI tasks by enhancing both the agent and reward models
  through reward-guided exploration and outcome verification in dynamic
  environments. For training the model, we generate UI- Genie-RM-517k and
  UI-Genie-Agent-16k, establishing the first reward-specific dataset for GUI
  agents while demonstrating high-quality synthetic trajectory gen- eration
  without manual annotation. Experimental results show that UI-Genie achieves
  state-of-the-art performance across multiple GUI agent benchmarks with three
  generations of data-model self-improvement. We open-source our complete
  framework implementation and generated datasets to facilitate further research
  in https://github.com/Euphoria16/UI-Genie.
paper_ai_summary: UI-Genie framework addresses GUI agent challenges through a reward model with image-text architecture and a self-improvement pipeline, achieving state-of-the-art performance on multiple benchmarks.
paper_ai_keywords: image-text interleaved architecture GUI agents reward model self-improving pipeline rule-based verification controlled trajectory corruption hard negative mining reward-guided exploration outcome verification dynamic environments synthetic trajectory generation