  The integration of information acquired with different modalities, spatial
resolution and spectral bands has shown to improve predictive accuracies. Data
fusion is therefore one of the key challenges in remote sensing. Most prior
work focusing on multi-modal fusion, assumes that modalities are always
available during inference. This assumption limits the applications of
multi-modal models since in practice the data collection process is likely to
generate data with missing, incomplete or corrupted modalities. In this paper,
we show that Generative Adversarial Networks can be effectively used to
overcome the problems that arise when modalities are missing or incomplete.
Focusing on semantic segmentation of building footprints with missing
modalities, our approach achieves an improvement of about 2% on the
Intersection over Union (IoU) against the same network that relies only on the
available modality.

Computer Vision and Pattern Recognition