Comparison of self-supervised in-domain and supervised out-domain transfer learning for bird species recognition
<|reference_start|>Comparison of self-supervised in-domain and supervised out-domain transfer learning for bird species recognition: Transferring the weights of a pre-trained model to assist another task has become a crucial part of modern deep learning, particularly in data-scarce scenarios. Pre-training refers to the initial step of training models outside the current task of interest, typically on another dataset. It can be done via supervised models using human-annotated datasets or self-supervised models trained on unlabeled datasets. In both cases, many pre-trained models are available to fine-tune for the task of interest. Interestingly, research has shown that pre-trained models from ImageNet can be helpful for audio tasks despite being trained on image datasets. Hence, it's unclear whether in-domain models would be advantageous compared to competent out-domain models, such as convolutional neural networks from ImageNet. Our experiments will demonstrate the usefulness of in-domain models and datasets for bird species recognition by leveraging VICReg, a recent and powerful self-supervised method.<|reference_end|>
@article{ghaffari2024comparison,
    title={Comparison of self-supervised in-domain and supervised out-domain
  transfer learning for bird species recognition},
    author={Houtan Ghaffari, Paul Devos},
    journal={arXiv preprint arXiv:2404.17252},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.17252},
    primaryClass={cs.LG cs.CV cs.SD}
}