  Nighttime thermal infrared (NTIR) image colorization, also known as
translation of NTIR images into daytime color images (NTIR2DC), is a promising
research direction to facilitate nighttime scene perception for humans and
intelligent systems under unfavorable conditions (e.g., complete darkness).
However, previously developed methods have poor colorization performance for
small sample classes. Moreover, reducing the high confidence noise in
pseudo-labels and addressing the problem of image gradient disappearance during
translation are still under-explored, and keeping edges from being distorted
during translation is also challenging. To address the aforementioned issues,
we propose a novel learning framework called Memory-guided cOllaboRative
atteNtion Generative Adversarial Network (MornGAN), which is inspired by the
analogical reasoning mechanisms of humans. Specifically, a memory-guided sample
selection strategy and adaptive collaborative attention loss are devised to
enhance the semantic preservation of small sample categories. In addition, we
propose an online semantic distillation module to mine and refine the
pseudo-labels of NTIR images. Further, conditional gradient repair loss is
introduced for reducing edge distortion during translation. Extensive
experiments on the NTIR2DC task show that the proposed MornGAN significantly
outperforms other image-to-image translation methods in terms of semantic
preservation and edge consistency, which helps improve the object detection
accuracy remarkably.

Computer Vision and Pattern Recognition