Semi-intrusive audio evaluation: Casting non-intrusive assessment as a multi-modal text prediction task
<|reference_start|>Semi-intrusive audio evaluation: Casting non-intrusive assessment as a multi-modal text prediction task: Assessment of audio by humans possesses the unique ability to attend to specific sources in a mixture of signals. Mimicking this human ability, we propose a semi-intrusive assessment where we frame the audio assessment task as a text prediction task with audio-text input. To this end we leverage instruction fine-tuning of the multi-modal PENGI model. Our experiments on MOS prediction for speech and music using both real and simulated data show that the proposed method, on average, outperforms baselines that operate on a single task. To justify the model generability, we propose a new semi-intrusive SNR estimator that is able to estimate the SNR of arbitrary signal classes in a mixture of signals with different classes.<|reference_end|>
@article{coldenhoff2024semi-intrusive,
    title={Semi-intrusive audio evaluation: Casting non-intrusive assessment as a
  multi-modal text prediction task},
    author={Jozef Coldenhoff, Milos Cernak},
    journal={arXiv preprint arXiv:2409.14069},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.14069},
    primaryClass={eess.AS cs.SD}
}