title: InfantAgent-Next: A Multimodal Generalist Agent for Automated Computer
    Interaction
summary: This paper introduces InfantAgent-Next, a generalist agent capable
  of interacting with computers in a multimodal manner, encompassing text,
  images, audio, and video. Unlike existing approaches that either build
  intricate workflows around a single large model or only provide workflow
  modularity, our agent integrates tool-based and pure vision agents within a
  highly modular architecture, enabling different models to collaboratively solve
  decoupled tasks in a step-by-step manner. Our generality is demonstrated by our
  ability to evaluate not only pure vision-based real-world benchmarks (i.e.,
  OSWorld), but also more general or tool-intensive benchmarks (e.g., GAIA and
  SWE-Bench). Specifically, we achieve 7.27% accuracy on OSWorld,
  higher than Claude-Computer-Use. Codes and evaluation scripts are open-sourced
  at https://github.com/bin123apple/InfantAgent.
paper_authors._id: 682b5387f1e88185bddb0643
paper_authors.name: Bin Lei
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-20T07:22:15.514Z
paper_authors._id: 648a2042e8bee533291da413
paper_authors.avatarUrl: /avatars/83b918ddd7e2130a1c72ae74606068dc.svg
paper_authors.fullname: Bin Lei
paper_authors.type: user
paper_authors.user: Bin12345
paper_authors._id: 682b5387f1e88185bddb0644
paper_authors.name: Weitai Kang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T12:51:00.774Z
paper_authors._id: 662050a24360f44332caf535
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/662050a24360f44332caf535/oVLyKvjCxgVV79taeRJPH.png
paper_authors.fullname: Weitai Kang
paper_authors.type: user
paper_authors.user: weitaikang
paper_authors._id: 682b5387f1e88185bddb0645
paper_authors.name: Zijian Zhang
paper_authors._id: 682b5387f1e88185bddb0646
paper_authors.name: Winson Chen
paper_authors._id: 682b5387f1e88185bddb0647
paper_authors.name: Xi Xie
paper_authors._id: 682b5387f1e88185bddb0648
paper_authors.name: Shan Zuo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T12:51:29.071Z
paper_authors._id: 649aab304952ac2c7a21f91c
paper_authors.avatarUrl: /avatars/d6e1df0dea2c922e47925a00fac696c5.svg
paper_authors.fullname: shanzuo
paper_authors.type: user
paper_authors.user: shanzuo
paper_authors._id: 682b5387f1e88185bddb0649
paper_authors.name: Mimi Xie
paper_authors._id: 682b5387f1e88185bddb064a
paper_authors.name: Ali Payani
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T12:51:42.477Z
paper_authors._id: 6411f7af46e5f8976912d62a
paper_authors.avatarUrl: /avatars/d8d5b012dc7704619fcd37b809b43059.svg
paper_authors.fullname: Ali Payani
paper_authors.type: user
paper_authors.user: apayani
paper_authors._id: 682b5387f1e88185bddb064b
paper_authors.name: Mingyi Hong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T12:51:48.788Z
paper_authors._id: 679a5fa807076ced8ae5f41b
paper_authors.avatarUrl: /avatars/63cac0a05c9c7500c9ab9c75667a38d7.svg
paper_authors.fullname: Mingyi Hong
paper_authors.type: user
paper_authors.user: poborskypaul
paper_authors._id: 682b5387f1e88185bddb064c
paper_authors.name: Yan Yan
paper_authors._id: 682b5387f1e88185bddb064d
paper_authors.name: Caiwen Ding
paper_title: InfantAgent-Next: A Multimodal Generalist Agent for Automated Computer
    Interaction
paper_summary: This paper introduces InfantAgent-Next, a generalist agent capable
  of interacting with computers in a multimodal manner, encompassing text,
  images, audio, and video. Unlike existing approaches that either build
  intricate workflows around a single large model or only provide workflow
  modularity, our agent integrates tool-based and pure vision agents within a
  highly modular architecture, enabling different models to collaboratively solve
  decoupled tasks in a step-by-step manner. Our generality is demonstrated by our
  ability to evaluate not only pure vision-based real-world benchmarks (i.e.,
  OSWorld), but also more general or tool-intensive benchmarks (e.g., GAIA and
  SWE-Bench). Specifically, we achieve 7.27% accuracy on OSWorld,
  higher than Claude-Computer-Use. Codes and evaluation scripts are open-sourced
  at https://github.com/bin123apple/InfantAgent.
paper_ai_summary: InfantAgent-Next is a multimodal agent that integrates tool-based and vision models in a modular architecture to solve various benchmarks, including OSWorld, GAIA, and SWE-Bench.
paper_ai_keywords: multimodal agent tool-based agents pure vision agents modular architecture OSWorld GAIA SWE-Bench