title: API Agents vs. GUI Agents: Divergence and Convergence
summary: Large language models (LLMs) have evolved beyond simple text generation to
  power software agents that directly translate natural language commands into
  tangible actions. While API-based LLM agents initially rose to prominence for
  their robust automation capabilities and seamless integration with programmatic
  endpoints, recent progress in multimodal LLM research has enabled GUI-based LLM
  agents that interact with graphical user interfaces in a human-like manner.
  Although these two paradigms share the goal of enabling LLM-driven task
  automation, they diverge significantly in architectural complexity, development
  workflows, and user interaction models.
    This paper presents the first comprehensive comparative study of API-based
  and GUI-based LLM agents, systematically analyzing their divergence and
  potential convergence. We examine key dimensions and highlight scenarios in
  which hybrid approaches can harness their complementary strengths. By proposing
  clear decision criteria and illustrating practical use cases, we aim to guide
  practitioners and researchers in selecting, combining, or transitioning between
  these paradigms. Ultimately, we indicate that continuing innovations in
  LLM-based automation are poised to blur the lines between API- and GUI-driven
  agents, paving the way for more flexible, adaptive solutions in a wide range of
  real-world applications.
paper_authors._id: 67d785458678eaf139e3c594
paper_authors.name: Chaoyun Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-17T08:51:30.401Z
paper_authors._id: 654dbac9938fbf1e696be8aa
paper_authors.avatarUrl: /avatars/b3c4035c48169c1bfb04a439fce3499f.svg
paper_authors.fullname: Chaoyun Zhang
paper_authors.type: user
paper_authors.user: vyokky
paper_authors._id: 67d785458678eaf139e3c595
paper_authors.name: Shilin He
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-17T08:51:37.539Z
paper_authors._id: 62c6df026a092eda1f1ab6e5
paper_authors.avatarUrl: /avatars/d58fff1a157b189ce2617889ef5f6e2f.svg
paper_authors.fullname: Shilin He
paper_authors.type: user
paper_authors.user: shilhe
paper_authors._id: 67d785458678eaf139e3c596
paper_authors.name: Liqun Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-17T08:51:57.886Z
paper_authors._id: 666933c97bf97e24f7b5266e
paper_authors.avatarUrl: /avatars/283961b37d463a386b08ad33dacca0f4.svg
paper_authors.fullname: Liqun Li
paper_authors.type: user
paper_authors.user: liqul
paper_authors._id: 67d785458678eaf139e3c597
paper_authors.name: Si Qin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-17T08:52:05.644Z
paper_authors._id: 67481846f47628abdd8c4397
paper_authors.avatarUrl: /avatars/b43f2988ac17bd2bb2369133934ce75d.svg
paper_authors.fullname: Si Qin
paper_authors.type: user
paper_authors.user: SiQin88
paper_authors._id: 67d785458678eaf139e3c598
paper_authors.name: Yu Kang
paper_authors._id: 67d785458678eaf139e3c599
paper_authors.name: Qingwei Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-17T08:52:17.826Z
paper_authors._id: 652fc9f39bc50a6c0e435224
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/652fc9f39bc50a6c0e435224/70OBVDHHBsxG2giJ-E3_1.jpeg
paper_authors.fullname: Lin Qingwei
paper_authors.type: user
paper_authors.user: Eliblo1969
paper_authors._id: 67d785458678eaf139e3c59a
paper_authors.name: Dongmei Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-17T08:52:31.623Z
paper_authors._id: 66473d2c7abe6ad66e81a3dd
paper_authors.avatarUrl: /avatars/82f40244806c06ffeaa1c4265e9725ea.svg
paper_authors.fullname: ZHANGDONGMEI
paper_authors.type: user
paper_authors.user: ZDM6426
paper_title: API Agents vs. GUI Agents: Divergence and Convergence
paper_summary: Large language models (LLMs) have evolved beyond simple text generation to
  power software agents that directly translate natural language commands into
  tangible actions. While API-based LLM agents initially rose to prominence for
  their robust automation capabilities and seamless integration with programmatic
  endpoints, recent progress in multimodal LLM research has enabled GUI-based LLM
  agents that interact with graphical user interfaces in a human-like manner.
  Although these two paradigms share the goal of enabling LLM-driven task
  automation, they diverge significantly in architectural complexity, development
  workflows, and user interaction models.
    This paper presents the first comprehensive comparative study of API-based
  and GUI-based LLM agents, systematically analyzing their divergence and
  potential convergence. We examine key dimensions and highlight scenarios in
  which hybrid approaches can harness their complementary strengths. By proposing
  clear decision criteria and illustrating practical use cases, we aim to guide
  practitioners and researchers in selecting, combining, or transitioning between
  these paradigms. Ultimately, we indicate that continuing innovations in
  LLM-based automation are poised to blur the lines between API- and GUI-driven
  agents, paving the way for more flexible, adaptive solutions in a wide range of
  real-world applications.