  Heterogeneous face re-identification, namely matching heterogeneous faces
across disjoint visible light (VIS) and near-infrared (NIR) cameras, has become
an important problem in video surveillance application. However, the large
domain discrepancy between heterogeneous NIR-VIS faces makes the performance of
face re-identification degraded dramatically. To solve this problem, a
multimodal fusion ranking optimization algorithm for heterogeneous face
re-identification is proposed in this paper. Firstly, we design a heterogeneous
face translation network to obtain multimodal face pairs, including
NIR-VIS/NIR-NIR/VIS-VIS face pairs, through mutual transformation between
NIR-VIS faces. Secondly, we propose linear and non-linear fusion strategies to
aggregate initial ranking lists of multimodal face pairs and acquire the
optimized re-ranked list based on modal complementarity. The experimental
results show that the proposed multimodal fusion ranking optimization algorithm
can effectively utilize the complementarity and outperforms some relative
methods on the SCface dataset.

Computer Vision and Pattern Recognition