title: Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and
    Dialogue Abilities
summary: Augmenting large language models (LLMs) to understand audio -- including
  non-speech sounds and non-verbal speech -- is critically important for diverse
  real-world applications of LLMs. In this paper, we propose Audio Flamingo, a
  novel audio language model with 1) strong audio understanding abilities, 2) the
  ability to quickly adapt to unseen tasks via in-context learning and retrieval,
  and 3) strong multi-turn dialogue abilities. We introduce a series of training
  techniques, architecture design, and data strategies to enhance our model with
  these abilities. Extensive evaluations across various audio understanding tasks
  confirm the efficacy of our method, setting new state-of-the-art benchmarks.
paper_authors._id: 65c1b5bf4f21259d993cfe92
paper_authors.name: Zhifeng Kong
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-07-17T06:56:05.962Z
paper_authors._id: 652a4dfc36f031c5e6f8b8a6
paper_authors.avatarUrl: /avatars/9fb56b025dc25f91ca6c31136eaf74b2.svg
paper_authors.fullname: Zhifeng Kong
paper_authors.type: user
paper_authors.user: ZhifengKong
paper_authors._id: 65c1b5bf4f21259d993cfe93
paper_authors.name: Arushi Goel
paper_authors._id: 65c1b5bf4f21259d993cfe94
paper_authors.name: Rohan Badlani
paper_authors._id: 65c1b5bf4f21259d993cfe95
paper_authors.name: Wei Ping
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-04-15T08:01:06.371Z
paper_authors._id: 661cdb08ed360bdc4f6db37c
paper_authors.avatarUrl: /avatars/bced64e9b7f46125a03d6f6e66ae7e33.svg
paper_authors.fullname: Wei Ping
paper_authors.type: user
paper_authors.user: wpingnet
paper_authors._id: 65c1b5bf4f21259d993cfe96
paper_authors.name: Rafael Valle
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-02-09T17:18:54.013Z
paper_authors._id: 67a8e3610836885c44bd2ab2
paper_authors.avatarUrl: /avatars/ae8d578a47de3302f07430924db3f105.svg
paper_authors.fullname: Rafael Valle
paper_authors.type: user
paper_authors.user: rafaelvallenvidia
paper_authors._id: 65c1b5bf4f21259d993cfe97
paper_authors.name: Bryan Catanzaro
paper_title: Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and
    Dialogue Abilities
paper_summary: Augmenting large language models (LLMs) to understand audio -- including
  non-speech sounds and non-verbal speech -- is critically important for diverse
  real-world applications of LLMs. In this paper, we propose Audio Flamingo, a
  novel audio language model with 1) strong audio understanding abilities, 2) the
  ability to quickly adapt to unseen tasks via in-context learning and retrieval,
  and 3) strong multi-turn dialogue abilities. We introduce a series of training
  techniques, architecture design, and data strategies to enhance our model with
  these abilities. Extensive evaluations across various audio understanding tasks
  confirm the efficacy of our method, setting new state-of-the-art benchmarks.
paper_ai_summary: Audio Flamingo, a novel audio language model, achieves strong audio understanding, in-context learning, and multi-turn dialogue capabilities through advanced training techniques and architecture design.
paper_ai_keywords: audio language model audio understanding in-context learning multi-turn dialogue training techniques architecture design