title: Audio-FLAN: A Preliminary Release
summary: Recent advancements in audio tokenization have significantly enhanced the
  integration of audio capabilities into large language models (LLMs). However,
  audio understanding and generation are often treated as distinct tasks,
  hindering the development of truly unified audio-language models. While
  instruction tuning has demonstrated remarkable success in improving
  generalization and zero-shot learning across text and vision, its application
  to audio remains largely unexplored. A major obstacle is the lack of
  comprehensive datasets that unify audio understanding and generation. To
  address this, we introduce Audio-FLAN, a large-scale instruction-tuning dataset
  covering 80 diverse tasks across speech, music, and sound domains, with over
  100 million instances. Audio-FLAN lays the foundation for unified
  audio-language models that can seamlessly handle both understanding (e.g.,
  transcription, comprehension) and generation (e.g., speech, music, sound) tasks
  across a wide range of audio domains in a zero-shot manner. The Audio-FLAN
  dataset is available on HuggingFace and GitHub and will be continuously
  updated.
paper_authors._id: 67bd42386959e61abd265a9b
paper_authors.name: Liumeng Xue
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-25T16:02:19.831Z
paper_authors._id: 6290e961473e457463a53248
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6290e961473e457463a53248/-58Dp5uHvdjs9yOupAMs0.jpeg
paper_authors.fullname: Liumeng Xue
paper_authors.type: user
paper_authors.user: lmxue
paper_authors._id: 67bd42386959e61abd265a9c
paper_authors.name: Ziya Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-25T16:04:58.842Z
paper_authors._id: 64191c925d6f3d15c65137b5
paper_authors.avatarUrl: /avatars/0e6a5fabf11904b9c31073ad1e10f6c6.svg
paper_authors.fullname: Ziya Zhou
paper_authors.type: user
paper_authors.user: DangeZy
paper_authors._id: 67bd42386959e61abd265a9d
paper_authors.name: Jiahao Pan
paper_authors._id: 67bd42386959e61abd265a9e
paper_authors.name: Zixuan Li
paper_authors._id: 67bd42386959e61abd265a9f
paper_authors.name: Shuai Fan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-25T16:05:22.700Z
paper_authors._id: 669a705f7c6ce7348dae9dcd
paper_authors.avatarUrl: /avatars/b042a8ea45848e3c9c77ce532286692f.svg
paper_authors.fullname: Shuai Fan
paper_authors.type: user
paper_authors.user: Micro20
paper_authors._id: 67bd42386959e61abd265aa0
paper_authors.name: Yinghao Ma
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-25T16:05:28.773Z
paper_authors._id: 6410665d5364a661bee22524
paper_authors.avatarUrl: /avatars/f1cb0e07f36933187ceccbd5dcbeff79.svg
paper_authors.fullname: Yinghao Ma
paper_authors.type: user
paper_authors.user: nicolaus625
paper_authors._id: 67bd42386959e61abd265aa1
paper_authors.name: Sitong Cheng
paper_authors._id: 67bd42386959e61abd265aa2
paper_authors.name: Dongchao Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-25T16:05:44.137Z
paper_authors._id: 63c7636b656e7822e23e6f6b
paper_authors.avatarUrl: /avatars/41bfd5e1ce6daab6058eacfd33c7a268.svg
paper_authors.fullname: Dongchao Yang
paper_authors.type: user
paper_authors.user: Dongchao
paper_authors._id: 67bd42386959e61abd265aa3
paper_authors.name: Haohan Guo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-25T16:05:50.198Z
paper_authors._id: 63783d9d84318944acd305c4
paper_authors.avatarUrl: /avatars/cafd4fd0287e77cc6ebf37c8c8509174.svg
paper_authors.fullname: Haohan Guo
paper_authors.type: user
paper_authors.user: hhguo
paper_authors._id: 67bd42386959e61abd265aa4
paper_authors.name: Yujia Xiao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-04T13:00:34.602Z
paper_authors._id: 674836767b7151c3ff30f865
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/jcwK5NW-efhCt8s2TE6vK.png
paper_authors.fullname: Yujia Xiao
paper_authors.type: user
paper_authors.user: Yogurt928
paper_authors._id: 67bd42386959e61abd265aa5
paper_authors.name: Xinsheng Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-25T16:06:03.447Z
paper_authors._id: 64cc69be8174e45ae076393e
paper_authors.avatarUrl: /avatars/6fd566535cadbed63e1f956587157d13.svg
paper_authors.fullname: Xinsheng Wang
paper_authors.type: user
paper_authors.user: wangxso
paper_authors._id: 67bd42386959e61abd265aa6
paper_authors.name: Zixuan Shen
paper_authors._id: 67bd42386959e61abd265aa7
paper_authors.name: Chuanbo Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-25T16:06:27.024Z
paper_authors._id: 6721db0fab7602a59648aec6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/bhWBJ26b_94mZICgi3jVZ.png
paper_authors.fullname: zhu chuanbo
paper_authors.type: user
paper_authors.user: zhuchb
paper_authors._id: 67bd42386959e61abd265aa8
paper_authors.name: Xinshen Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-25T16:06:40.621Z
paper_authors._id: 668b90710b04331a0bbacbb0
paper_authors.avatarUrl: /avatars/1a457fc4b4d242a9a7104ba38d5c2467.svg
paper_authors.fullname: ZHANG Xinshen
paper_authors.type: user
paper_authors.user: Ashaire
paper_authors._id: 67bd42386959e61abd265aa9
paper_authors.name: Tianchi Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-25T16:06:47.569Z
paper_authors._id: 6756ae32298969739a42d5f9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6756ae32298969739a42d5f9/XethqlkSvZUi3ZFnNLqaF.jpeg
paper_authors.fullname: Tianchi Liu
paper_authors.type: user
paper_authors.user: liu-tianchi
paper_authors._id: 67bd42386959e61abd265aaa
paper_authors.name: Ruibin Yuan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-25T16:06:54.742Z
paper_authors._id: 5fd6f670053c8345eddc1b68
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/5fd6f670053c8345eddc1b68/cuTsu2krRYHC6zYGD2dpQ.jpeg
paper_authors.fullname: Ruibin Yuan
paper_authors.type: user
paper_authors.user: a43992899
paper_authors._id: 67bd42386959e61abd265aab
paper_authors.name: Zeyue Tian
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-16T21:18:50.821Z
paper_authors._id: 6504409b52ca06fef957266d
paper_authors.avatarUrl: /avatars/ac96eab7318cd8d472601cf7e7c50c97.svg
paper_authors.fullname: Zeyue
paper_authors.type: user
paper_authors.user: Zeyue7
paper_authors._id: 67bd42386959e61abd265aac
paper_authors.name: Haohe Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-25T16:07:13.402Z
paper_authors._id: 6155245d1c762d4d61b51d5d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1677233708065-6155245d1c762d4d61b51d5d.png
paper_authors.fullname: haoheliu
paper_authors.type: user
paper_authors.user: haoheliu
paper_authors._id: 67bd42386959e61abd265aad
paper_authors.name: Emmanouil Benetos
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-25T16:07:19.685Z
paper_authors._id: 66b0a0839797b94f001ed874
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/66b0a0839797b94f001ed874/BBtlJdQNyzQX10chnU81s.jpeg
paper_authors.fullname: Emmanouil Benetos
paper_authors.type: user
paper_authors.user: emmanouilb
paper_authors._id: 67bd42386959e61abd265aae
paper_authors.name: Ge Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-25T16:07:26.226Z
paper_authors._id: 638efcf4c67af472d316d424
paper_authors.avatarUrl: /avatars/97a57859d7d87a3a8f1bb41d32a72bc2.svg
paper_authors.fullname: Ge Zhang
paper_authors.type: user
paper_authors.user: zhangysk
paper_authors._id: 67bd42386959e61abd265aaf
paper_authors.name: Yike Guo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-25T16:07:36.243Z
paper_authors._id: 64762b325f70f9b2d0ade28e
paper_authors.avatarUrl: /avatars/1f37382724b475ece805f943a8858acd.svg
paper_authors.fullname: Yike Guo
paper_authors.type: user
paper_authors.user: SuaLily
paper_authors._id: 67bd42386959e61abd265ab0
paper_authors.name: Wei Xue
paper_title: Audio-FLAN: A Preliminary Release
paper_summary: Recent advancements in audio tokenization have significantly enhanced the
  integration of audio capabilities into large language models (LLMs). However,
  audio understanding and generation are often treated as distinct tasks,
  hindering the development of truly unified audio-language models. While
  instruction tuning has demonstrated remarkable success in improving
  generalization and zero-shot learning across text and vision, its application
  to audio remains largely unexplored. A major obstacle is the lack of
  comprehensive datasets that unify audio understanding and generation. To
  address this, we introduce Audio-FLAN, a large-scale instruction-tuning dataset
  covering 80 diverse tasks across speech, music, and sound domains, with over
  100 million instances. Audio-FLAN lays the foundation for unified
  audio-language models that can seamlessly handle both understanding (e.g.,
  transcription, comprehension) and generation (e.g., speech, music, sound) tasks
  across a wide range of audio domains in a zero-shot manner. The Audio-FLAN
  dataset is available on HuggingFace and GitHub and will be continuously
  updated.
paper_ai_keywords: instruction tuning generalization zero-shot learning audio understanding audio generation comprehensive datasets Audio-FLAN diverse tasks speech music sound domains transcription comprehension unified audio-language models