Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and
  Dialogue Abilities
Augmenting large language models (LLMs) to understand audio -- including
non-speech sounds and non-verbal speech -- is critically important for diverse
real-world applications of LLMs. In this paper, we propose Audio Flamingo, a
novel audio language model with 1) strong audio understanding abilities, 2) the
ability to quickly adapt to unseen tasks via in-context learning and retrieval,
and 3) strong multi-turn dialogue abilities. We introduce a series of training
techniques, architecture design, and data strategies to enhance our model with
these abilities. Extensive evaluations across various audio understanding tasks
confirm the efficacy of our method, setting new state-of-the-art benchmarks.
65c1b5bf4f21259d993cfe92
Zhifeng Kong
extracted_pending
2024-07-17T06:56:05.962Z
652a4dfc36f031c5e6f8b8a6
/avatars/9fb56b025dc25f91ca6c31136eaf74b2.svg
Zhifeng Kong
user
ZhifengKong
65c1b5bf4f21259d993cfe93
Arushi Goel
65c1b5bf4f21259d993cfe94
Rohan Badlani
65c1b5bf4f21259d993cfe95
Wei Ping
claimed_verified
2024-04-15T08:01:06.371Z
661cdb08ed360bdc4f6db37c
/avatars/bced64e9b7f46125a03d6f6e66ae7e33.svg
Wei Ping
user
wpingnet
65c1b5bf4f21259d993cfe96
Rafael Valle
extracted_pending
2025-02-09T17:18:54.013Z
67a8e3610836885c44bd2ab2
/avatars/ae8d578a47de3302f07430924db3f105.svg
Rafael Valle
user
rafaelvallenvidia
65c1b5bf4f21259d993cfe97
Bryan Catanzaro
Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and
  Dialogue Abilities
Augmenting large language models (LLMs) to understand audio -- including
non-speech sounds and non-verbal speech -- is critically important for diverse
real-world applications of LLMs. In this paper, we propose Audio Flamingo, a
novel audio language model with 1) strong audio understanding abilities, 2) the
ability to quickly adapt to unseen tasks via in-context learning and retrieval,
and 3) strong multi-turn dialogue abilities. We introduce a series of training
techniques, architecture design, and data strategies to enhance our model with
these abilities. Extensive evaluations across various audio understanding tasks
confirm the efficacy of our method, setting new state-of-the-art benchmarks.
Audio Flamingo, a novel audio language model, achieves strong audio understanding, in-context learning, and multi-turn dialogue capabilities through advanced training techniques and architecture design.
audio language model audio understanding in-context learning multi-turn dialogue training techniques architecture design