To validate its effectiveness, we tested our agent
on 50 tasks across 10 different apps, ranging from
social media and messaging to email, maps, shop-
ping, and even complex image editing apps. Both
quantitative results and user studies underscore the
advantages of our design, particularly its adapt-
ability, user-friendliness, and efficient learning and

2

operating capabilities across a wide range of appli-
cations. This underlines the potential of our agent
as a versatile and effective tool in the realm of
smartphone app operation.

In summary, this paper makes the following con-

tributions:

2023b,a), ChartLlama (Han et al., 2023), and Sta-
bleLLaVA (Li et al., 2023), also demonstrate vision
understanding capabilities akin to those of GPT-4V.
Nevertheless, a performance gap persists between
these open-source models and GPT-4V, suggesting
potential areas for further development.
AppAgents