  We present a novel method to estimate the surface normal of an object in an
ambient light environment using RGB and event cameras. Modern photometric
stereo methods rely on an RGB camera, mainly in a dark room, to avoid ambient
illumination. To alleviate the limitations of the darkroom environment and to
use essential light information, we employ an event camera with a high dynamic
range and low latency. This is the first study that uses an event camera for
the photometric stereo task, which works on continuous light sources and
ambient light environment. In this work, we also curate a novel photometric
stereo dataset that is constructed by capturing objects with event and RGB
cameras under numerous ambient lights environment. Additionally, we propose a
novel framework named Event Fusion Photometric Stereo Network~(EFPS-Net), which
estimates the surface normals of an object using both RGB frames and event
signals. Our proposed method interpolates event observation maps that generate
light information with sparse event signals to acquire fluent light
information. Subsequently, the event-interpolated observation maps are fused
with the RGB observation maps. Our numerous experiments showed that EFPS-Net
outperforms state-of-the-art methods on a dataset captured in the real world
where ambient lights exist. Consequently, we demonstrate that incorporating
additional modalities with EFPS-Net alleviates the limitations that occurred
from ambient illumination.

Computer Vision and Pattern Recognition