  Remote sensing (RS) image scene classification task faces many challenges due
to the interference from different characteristics of different geographical
elements. To solve this problem, we propose a multi-branch ensemble network to
enhance the feature representation ability by fusing features in final output
logits and intermediate feature maps. However, simply adding branches will
increase the complexity of models and decline the inference efficiency. On this
issue, we embed self-distillation (SD) method to transfer knowledge from
ensemble network to main-branch in it. Through optimizing with SD, main-branch
will have close performance as ensemble network. During inference, we can cut
other branches to simplify the whole model. In this paper, we first design
compact multi-branch ensemble network, which can be trained in an end-to-end
manner. Then, we insert SD method on output logits and feature maps. Compared
to previous methods, our proposed architecture (ESD-MBENet) performs strongly
on classification accuracy with compact design. Extensive experiments are
applied on three benchmark RS datasets AID, NWPU-RESISC45 and UC-Merced with
three classic baseline models, VGG16, ResNet50 and DenseNet121. Results prove
that our proposed ESD-MBENet can achieve better accuracy than previous
state-of-the-art (SOTA) complex models. Moreover, abundant visualization
analysis make our method more convincing and interpretable.

Computer Vision and Pattern Recognition;Machine Learning