Quantitative Analysis of Audio-Visual Tasks: An Information-Theoretic Perspective
<|reference_start|>Quantitative Analysis of Audio-Visual Tasks: An Information-Theoretic Perspective: In the field of spoken language processing, audio-visual speech processing is receiving increasing research attention. Key components of this research include tasks such as lip reading, audio-visual speech recognition, and visual-to-speech synthesis. Although significant success has been achieved, theoretical analysis is still insufficient for audio-visual tasks. This paper presents a quantitative analysis based on information theory, focusing on information intersection between different modalities. Our results show that this analysis is valuable for understanding the difficulties of audio-visual processing tasks as well as the benefits that could be obtained by modality integration.<|reference_end|>
@article{chen2024quantitative,
    title={Quantitative Analysis of Audio-Visual Tasks: An Information-Theoretic
  Perspective},
    author={Chen Chen, Xiaolou Li, Zehua Liu, Lantian Li, Dong Wang},
    journal={arXiv preprint arXiv:2409.19575},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.19575},
    primaryClass={cs.SD cs.CL cs.MM eess.AS}
}