Dyn-Adapter: Towards Disentangled Representation for Efficient Visual Recognition
<|reference_start|>Dyn-Adapter: Towards Disentangled Representation for Efficient Visual Recognition: Parameter-efficient transfer learning (PETL) is a promising task, aiming to adapt the large-scale pre-trained model to downstream tasks with a relatively modest cost. However, current PETL methods struggle in compressing computational complexity and bear a heavy inference burden due to the complete forward process. This paper presents an efficient visual recognition paradigm, called Dynamic Adapter (Dyn-Adapter), that boosts PETL efficiency by subtly disentangling features in multiple levels. Our approach is simple: first, we devise a dynamic architecture with balanced early heads for multi-level feature extraction, along with adaptive training strategy. Second, we introduce a bidirectional sparsity strategy driven by the pursuit of powerful generalization ability. These qualities enable us to fine-tune efficiently and effectively: we reduce FLOPs during inference by 50%, while maintaining or even yielding higher recognition accuracy. Extensive experiments on diverse datasets and pretrained backbones demonstrate the potential of Dyn-Adapter serving as a general efficiency booster for PETL in vision recognition tasks.<|reference_end|>
@article{zhang2024dyn-adapter:,
    title={Dyn-Adapter: Towards Disentangled Representation for Efficient Visual
  Recognition},
    author={Yurong Zhang, Honghao Chen, Xinyu Zhang, Xiangxiang Chu, Li Song},
    journal={arXiv preprint arXiv:2407.14302},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.14302},
    primaryClass={cs.CV}
}