Robust Fine-tuning for Pre-trained 3D Point Cloud Models
<|reference_start|>Robust Fine-tuning for Pre-trained 3D Point Cloud Models: This paper presents a robust fine-tuning method designed for pre-trained 3D point cloud models, to enhance feature robustness in downstream fine-tuned models. We highlight the limitations of current fine-tuning methods and the challenges of learning robust models. The proposed method, named Weight-Space Ensembles for Fine-Tuning then Linear Probing (WiSE-FT-LP), integrates the original pre-training and fine-tuning models through weight space integration followed by Linear Probing. This approach significantly enhances the performance of downstream fine-tuned models under distribution shifts, improving feature robustness while maintaining high performance on the target distribution. We apply this robust fine-tuning method to mainstream 3D point cloud pre-trained models and evaluate the quality of model parameters and the degradation of downstream task performance. Experimental results demonstrate the effectiveness of WiSE-FT-LP in enhancing model robustness, effectively balancing downstream task performance and model feature robustness without altering the model structures.<|reference_end|>
@article{zhang2024robust,
    title={Robust Fine-tuning for Pre-trained 3D Point Cloud Models},
    author={Zhibo Zhang, Ximing Yang, Weizhong Zhang, Cheng Jin},
    journal={arXiv preprint arXiv:2404.16422},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.16422},
    primaryClass={cs.CV}
}