  Traditional visual place recognition (VPR), usually using standard cameras,
is easy to fail due to glare or high-speed motion. By contrast, event cameras
have the advantages of low latency, high temporal resolution, and high dynamic
range, which can deal with the above issues. Nevertheless, event cameras are
prone to failure in weakly textured or motionless scenes, while standard
cameras can still provide appearance information in this case. Thus, exploiting
the complementarity of standard cameras and event cameras can effectively
improve the performance of VPR algorithms. In the paper, we propose
FE-Fusion-VPR, an attention-based multi-scale network architecture for VPR by
fusing frames and events. First, the intensity frame and event volume are fed
into the two-stream feature extraction network for shallow feature fusion.
Next, the three-scale features are obtained through the multi-scale fusion
network and aggregated into three sub-descriptors using the VLAD layer.
Finally, the weight of each sub-descriptor is learned through the descriptor
re-weighting network to obtain the final refined descriptor. Experimental
results show that on the Brisbane-Event-VPR and DDD20 datasets, the Recall@1 of
our FE-Fusion-VPR is 29.26% and 33.59% higher than Event-VPR and
Ensemble-EventVPR, and is 7.00% and 14.15% higher than MultiRes-NetVLAD and
NetVLAD. To our knowledge, this is the first end-to-end network that goes
beyond the existing event-based and frame-based SOTA methods to fuse frame and
events directly for VPR.

Computer Vision and Pattern Recognition;Robotics