  Recently, learning-based algorithms have achieved promising performance on
cross-spectral image patch matching, which, however, is still far from
satisfactory for practical application. On the one hand, a lack of large-scale
dataset with diverse scenes haunts its further improvement for learning-based
algorithms, whose performances and generalization rely heavily on the dataset
size and diversity. On the other hand, more emphasis has been put on feature
relation in the spatial domain whereas the scale dependency between features
has often been ignored, leading to performance degeneration especially when
encountering significant appearance variations for cross-spectral patches. To
address these issues, we publish, to be best of our knowledge, the largest
visible and Long-wave Infrared (LWIR) image patch matching dataset, termed
VL-CMIM, which contains 1300 pairs of strictly aligned visible and LWIR images
and over 2 million patch pairs covering diverse scenes such as asteroid, field,
country, build, street and water.In addition, a multi-domain feature relation
learning network (MD-FRN) is proposed. Input by the features extracted from a
four-branch network, both feature relations in spatial and scale domains are
learned via a spatial correlation module (SCM) and multi-scale adaptive
aggregation module (MSAG), respectively. To further aggregate the multi-domain
relations, a deep domain interactive mechanism (DIM) is applied, where the
learnt spatial-relation and scale-relation features are exchanged and further
input into MSCRM and SCM. This mechanism allows our model to learn interactive
cross-domain feature relations, leading to improved robustness to significant
appearance changes due to different modality.

Computer Vision and Pattern Recognition