InfantAgent-Next: A Multimodal Generalist Agent for Automated Computer
  Interaction
This paper introduces InfantAgent-Next, a generalist agent capable
of interacting with computers in a multimodal manner, encompassing text,
images, audio, and video. Unlike existing approaches that either build
intricate workflows around a single large model or only provide workflow
modularity, our agent integrates tool-based and pure vision agents within a
highly modular architecture, enabling different models to collaboratively solve
decoupled tasks in a step-by-step manner. Our generality is demonstrated by our
ability to evaluate not only pure vision-based real-world benchmarks (i.e.,
OSWorld), but also more general or tool-intensive benchmarks (e.g., GAIA and
SWE-Bench). Specifically, we achieve 7.27% accuracy on OSWorld,
higher than Claude-Computer-Use. Codes and evaluation scripts are open-sourced
at https://github.com/bin123apple/InfantAgent.
682b5387f1e88185bddb0643
Bin Lei
claimed_verified
2025-05-20T07:22:15.514Z
648a2042e8bee533291da413
/avatars/83b918ddd7e2130a1c72ae74606068dc.svg
Bin Lei
user
Bin12345
682b5387f1e88185bddb0644
Weitai Kang
admin_assigned
2025-05-27T12:51:00.774Z
662050a24360f44332caf535
https://cdn-avatars.huggingface.co/v1/production/uploads/662050a24360f44332caf535/oVLyKvjCxgVV79taeRJPH.png
Weitai Kang
user
weitaikang
682b5387f1e88185bddb0645
Zijian Zhang
682b5387f1e88185bddb0646
Winson Chen
682b5387f1e88185bddb0647
Xi Xie
682b5387f1e88185bddb0648
Shan Zuo
admin_assigned
2025-05-27T12:51:29.071Z
649aab304952ac2c7a21f91c
/avatars/d6e1df0dea2c922e47925a00fac696c5.svg
shanzuo
user
shanzuo
682b5387f1e88185bddb0649
Mimi Xie
682b5387f1e88185bddb064a
Ali Payani
admin_assigned
2025-05-27T12:51:42.477Z
6411f7af46e5f8976912d62a
/avatars/d8d5b012dc7704619fcd37b809b43059.svg
Ali Payani
user
apayani
682b5387f1e88185bddb064b
Mingyi Hong
admin_assigned
2025-05-27T12:51:48.788Z
679a5fa807076ced8ae5f41b
/avatars/63cac0a05c9c7500c9ab9c75667a38d7.svg
Mingyi Hong
user
poborskypaul
682b5387f1e88185bddb064c
Yan Yan
682b5387f1e88185bddb064d
Caiwen Ding
InfantAgent-Next: A Multimodal Generalist Agent for Automated Computer
  Interaction
This paper introduces InfantAgent-Next, a generalist agent capable
of interacting with computers in a multimodal manner, encompassing text,
images, audio, and video. Unlike existing approaches that either build
intricate workflows around a single large model or only provide workflow
modularity, our agent integrates tool-based and pure vision agents within a
highly modular architecture, enabling different models to collaboratively solve
decoupled tasks in a step-by-step manner. Our generality is demonstrated by our
ability to evaluate not only pure vision-based real-world benchmarks (i.e.,
OSWorld), but also more general or tool-intensive benchmarks (e.g., GAIA and
SWE-Bench). Specifically, we achieve 7.27% accuracy on OSWorld,
higher than Claude-Computer-Use. Codes and evaluation scripts are open-sourced
at https://github.com/bin123apple/InfantAgent.
InfantAgent-Next is a multimodal agent that integrates tool-based and vision models in a modular architecture to solve various benchmarks, including OSWorld, GAIA, and SWE-Bench.
multimodal agent tool-based agents pure vision agents modular architecture OSWorld GAIA SWE-Bench