  This work presents MemX: a biologically-inspired attention-aware eyewear
system developed with the goal of pursuing the long-awaited vision of a
personalized visual Memex. MemX captures human visual attention on the fly,
analyzes the salient visual content, and records moments of personal interest
in the form of compact video snippets. Accurate attentive scene detection and
analysis on resource-constrained platforms is challenging because these tasks
are computation and energy intensive. We propose a new temporal visual
attention network that unifies human visual attention tracking and salient
visual content analysis. Attention tracking focuses computation-intensive video
analysis on salient regions, while video analysis makes human attention
detection and tracking more accurate. Using the YouTube-VIS dataset and 30
participants, we experimentally show that MemX significantly improves the
attention tracking accuracy over the eye-tracking-alone method, while
maintaining high system energy efficiency. We have also conducted 11 in-field
pilot studies across a range of daily usage scenarios, which demonstrate the
feasibility and potential benefits of MemX.

Computer Vision and Pattern Recognition;Human-Computer Interaction