  We introduce AirWare, an in-air hand-gesture recognition system that uses the
already embedded speaker and microphone in most electronic devices, together
with embedded infrared proximity sensors. Gestures identified by AirWare are
performed in the air above a touchscreen or a mobile phone. AirWare utilizes
convolutional neural networks to classify a large vocabulary of hand gestures
using multi-modal audio Doppler signatures and infrared (IR) sensor
information. As opposed to other systems which use high frequency Doppler
radars or depth cameras to uniquely identify in-air gestures, AirWare does not
require any external sensors. In our analysis, we use openly available APIs to
interface with the Samsung Galaxy S5 audio and proximity sensors for data
collection. We find that AirWare is not reliable enough for a deployable
interaction system when trying to classify a gesture set of 21 gestures, with
an average true positive rate of only 50.5% per gesture. To improve
performance, we train AirWare to identify subsets of the 21 gestures vocabulary
based on possible usage scenarios. We find that AirWare can identify three
gesture sets with average true positive rate greater than 80% using 4--7
gestures per set, which comprises a vocabulary of 16 unique in-air gestures.

Human-Computer Interaction