  The challenge stated in the title can be divided into two main problems. The
first problem is to reliably mimic the way that users interact with user
interfaces. The second problem is to build an instructible agent, i.e. one that
can be taught to execute tasks expressed as previously unseen natural language
commands. This paper proposes a solution to the second problem, a system we
call Helpa. End-users can teach Helpa arbitrary new tasks whose level of
complexity is similar to the tasks available from today's most popular virtual
assistants. Teaching Helpa does not involve any programming. Instead, users
teach Helpa by providing just one example of a command paired with a
demonstration of how to execute that command. Helpa does not rely on any
pre-existing domain-specific knowledge. It is therefore completely
domain-independent. Our usability study showed that end-users can teach Helpa
many new tasks in less than a minute each, often much less.

Artificial Intelligence