Audio Dialogues: Dialogues dataset for audio and music understanding
Existing datasets for audio understanding primarily focus on single-turn
interactions (i.e. audio captioning, audio question answering) for describing
audio in natural language, thus limiting understanding audio via interactive
dialogue. To address this gap, we introduce Audio Dialogues: a multi-turn
dialogue dataset containing 163.8k samples for general audio sounds and music.
In addition to dialogues, Audio Dialogues also has question-answer pairs to
understand and compare multiple input audios together. Audio Dialogues
leverages a prompting-based approach and caption annotations from existing
datasets to generate multi-turn dialogues using a Large Language Model (LLM).
We evaluate existing audio-augmented large language models on our proposed
dataset to demonstrate the complexity and applicability of Audio Dialogues. Our
code for generating the dataset will be made publicly available. Detailed
prompts and generated dialogues can be found on the demo website
https://audiodialogues.github.io/.
66188dbbc69a25d5e72040b5
Arushi Goel
extracted_pending
2024-04-12T01:26:19.902Z
65c0b5babf441a1953586df7
/avatars/b767ab08dc096e0aeac29acfcb26d39e.svg
Arushi Goel
user
goarushi27
66188dbbc69a25d5e72040b6
Zhifeng Kong
admin_assigned
2024-04-18T12:27:34.590Z
652a4dfc36f031c5e6f8b8a6
/avatars/9fb56b025dc25f91ca6c31136eaf74b2.svg
Zhifeng Kong
user
ZhifengKong
66188dbbc69a25d5e72040b7
Rafael Valle
admin_assigned
2024-04-18T12:27:52.372Z
6440ddd65d600fb09518daa8
/avatars/ac5898afd2082d230e2ebf6fb867ad4f.svg
Rafael Valle
user
rafaelvalle
66188dbbc69a25d5e72040b8
Bryan Catanzaro
admin_assigned
2024-04-18T12:27:58.405Z
6311021788942700629e6247
/avatars/e7adc1632b76e80e7e4a590033d1c20a.svg
Bryan Catanzaro
user
ctnzr
Audio Dialogues: Dialogues dataset for audio and music understanding
Existing datasets for audio understanding primarily focus on single-turn
interactions (i.e. audio captioning, audio question answering) for describing
audio in natural language, thus limiting understanding audio via interactive
dialogue. To address this gap, we introduce Audio Dialogues: a multi-turn
dialogue dataset containing 163.8k samples for general audio sounds and music.
In addition to dialogues, Audio Dialogues also has question-answer pairs to
understand and compare multiple input audios together. Audio Dialogues
leverages a prompting-based approach and caption annotations from existing
datasets to generate multi-turn dialogues using a Large Language Model (LLM).
We evaluate existing audio-augmented large language models on our proposed
dataset to demonstrate the complexity and applicability of Audio Dialogues. Our
code for generating the dataset will be made publicly available. Detailed
prompts and generated dialogues can be found on the demo website
https://audiodialogues.github.io/.
A new multi-turn dialogue dataset for audio, Audio Dialogues, is introduced to enhance understanding and dialogue capabilities in audio processing using a Large Language Model.
audio captioning audio question answering multi-turn dialogue Large Language Model audio-augmented large language models