Effective Pre-Training of Audio Transformers for Sound Event Detection
<|reference_start|>Effective Pre-Training of Audio Transformers for Sound Event Detection: We propose a pre-training pipeline for audio spectrogram transformers for frame-level sound event detection tasks. On top of common pre-training steps, we add a meticulously designed training routine on AudioSet frame-level annotations. This includes a balanced sampler, aggressive data augmentation, and ensemble knowledge distillation. For five transformers, we obtain a substantial performance improvement over previously available checkpoints both on AudioSet frame-level predictions and on frame-level sound event detection downstream tasks, confirming our pipeline's effectiveness. We publish the resulting checkpoints that researchers can directly fine-tune to build high-performance models for sound event detection tasks.<|reference_end|>
@article{schmid2024effective,
    title={Effective Pre-Training of Audio Transformers for Sound Event Detection},
    author={Florian Schmid, Tobias Morocutti, Francesco Foscarin, Jan Schl"uter,
  Paul Primus, Gerhard Widmer},
    journal={arXiv preprint arXiv:2409.09546},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.09546},
    primaryClass={eess.AS cs.SD}
}