  Augmented reality devices require multiple sensors to perform various tasks
such as localization and tracking. Currently, popular cameras are mostly
frame-based (e.g. RGB and Depth) which impose a high data bandwidth and power
usage. With the necessity for low power and more responsive augmented reality
systems, using solely frame-based sensors imposes limits to the various
algorithms that needs high frequency data from the environement. As such,
event-based sensors have become increasingly popular due to their low power,
bandwidth and latency, as well as their very high frequency data acquisition
capabilities. In this paper, we propose, for the first time, to use an
event-based camera to increase the speed of 3D object tracking in 6 degrees of
freedom. This application requires handling very high object speed to convey
compelling AR experiences. To this end, we propose a new system which combines
a recent RGB-D sensor (Kinect Azure) with an event camera (DAVIS346). We
develop a deep learning approach, which combines an existing RGB-D network
along with a novel event-based network in a cascade fashion, and demonstrate
that our approach significantly improves the robustness of a state-of-the-art
frame-based 6-DOF object tracker using our RGB-D-E pipeline.

Computer Vision and Pattern Recognition