SingOMD: Singing Oriented Multi-resolution Discrete Representation Construction from Speech Models
<|reference_start|>SingOMD: Singing Oriented Multi-resolution Discrete Representation Construction from Speech Models: Discrete representation has shown advantages in speech generation tasks, wherein discrete tokens are derived by discretizing hidden features from self-supervised learning (SSL) pre-trained models. However, the direct application of speech SSL models to singing generation encounters domain gaps between speech and singing. Furthermore, singing generation necessitates a more refined representation than typical speech. To address these challenges, we introduce SingOMD, a novel method to extract singing-oriented multi-resolution discrete representations from speech SSL models. Specifically, we first adapt the features from speech SSL through a resynthesis task and incorporate multi-resolution modules based on resampling to better serve singing generation. These adapted multi-resolution features are then discretized via clustering. Extensive experiments demonstrate the robustness, efficiency, and effectiveness of these representations in singing vocoders and singing voice synthesis.<|reference_end|>
@article{tang2024singomd:,
    title={SingOMD: Singing Oriented Multi-resolution Discrete Representation
  Construction from Speech Models},
    author={Yuxun Tang, Yuning Wu, Jiatong Shi, Qin Jin},
    journal={arXiv preprint arXiv:2406.08905},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.08905},
    primaryClass={cs.SD eess.AS}
}