  Nowadays, voice assistants help users complete tasks on the smartphone with
voice commands, replacing traditional touchscreen interactions when such
interactions are inhibited. However, the usability of those tools remains
moderate due to the problems in understanding rich language variations in human
commands, along with efficiency and comprehensibility issues. Therefore, we
introduce Voicify, an Android virtual assistant that allows users to interact
with on-screen elements in mobile apps through voice commands. Using a novel
deep learning command parser, Voicify interprets human verbal input and
performs matching with UI elements. In addition, the tool can directly open a
specific feature from installed applications by fetching application code
information to explore the set of in-app components. Our command parser
achieved 90\% accuracy on the human command dataset. Furthermore, the direct
feature invocation module achieves better feature coverage in comparison to
Google Assistant. The user study demonstrates the usefulness of Voicify in
real-world scenarios.

Human-Computer Interaction