title: LP-MusicCaps: LLM-Based Pseudo Music Captioning
summary: Automatic music captioning, which generates natural language descriptions for
  given music tracks, holds significant potential for enhancing the understanding
  and organization of large volumes of musical data. Despite its importance,
  researchers face challenges due to the costly and time-consuming collection
  process of existing music-language datasets, which are limited in size. To
  address this data scarcity issue, we propose the use of large language models
  (LLMs) to artificially generate the description sentences from large-scale tag
  datasets. This results in approximately 2.2M captions paired with 0.5M audio
  clips. We term it Large Language Model based Pseudo music caption dataset,
  shortly, LP-MusicCaps. We conduct a systemic evaluation of the large-scale
  music captioning dataset with various quantitative evaluation metrics used in
  the field of natural language processing as well as human evaluation. In
  addition, we trained a transformer-based music captioning model with the
  dataset and evaluated it under zero-shot and transfer-learning settings. The
  results demonstrate that our proposed approach outperforms the supervised
  baseline model.
paper_authors._id: 64c8694e3cfe45b07147b1ad
paper_authors.name: SeungHeon Doh
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-08-01T07:44:29.602Z
paper_authors._id: 637c3504c292c0fd3f37361f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/637c3504c292c0fd3f37361f/wyTkbYKi8HufRT65LGN0P.jpeg
paper_authors.fullname: seungheon.doh
paper_authors.type: user
paper_authors.user: seungheondoh
paper_authors._id: 64c8694e3cfe45b07147b1ae
paper_authors.name: Keunwoo Choi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-01T09:24:50.173Z
paper_authors._id: 63c4c4bbc7d7f4c63a624db1
paper_authors.avatarUrl: /avatars/b701e14f222aef3d3c52658b5b95c98e.svg
paper_authors.fullname: Keunwoo choi
paper_authors.type: user
paper_authors.user: keunwoochoi
paper_authors._id: 64c8694e3cfe45b07147b1af
paper_authors.name: Jongpil Lee
paper_authors._id: 64c8694e3cfe45b07147b1b0
paper_authors.name: Juhan Nam
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-08-07T12:38:57.409Z
paper_authors._id: 64cf3e0c9c245c6ba728c831
paper_authors.avatarUrl: /avatars/740b7c3a25568b5556cc34104322209f.svg
paper_authors.fullname: Juhan Nam
paper_authors.type: user
paper_authors.user: juhannam
paper_title: LP-MusicCaps: LLM-Based Pseudo Music Captioning
paper_summary: Automatic music captioning, which generates natural language descriptions for
  given music tracks, holds significant potential for enhancing the understanding
  and organization of large volumes of musical data. Despite its importance,
  researchers face challenges due to the costly and time-consuming collection
  process of existing music-language datasets, which are limited in size. To
  address this data scarcity issue, we propose the use of large language models
  (LLMs) to artificially generate the description sentences from large-scale tag
  datasets. This results in approximately 2.2M captions paired with 0.5M audio
  clips. We term it Large Language Model based Pseudo music caption dataset,
  shortly, LP-MusicCaps. We conduct a systemic evaluation of the large-scale
  music captioning dataset with various quantitative evaluation metrics used in
  the field of natural language processing as well as human evaluation. In
  addition, we trained a transformer-based music captioning model with the
  dataset and evaluated it under zero-shot and transfer-learning settings. The
  results demonstrate that our proposed approach outperforms the supervised
  baseline model.
paper_ai_keywords: neighborhood language models (LLMs) Large Language Model based Pseudo music caption dataset (LP-MusicCaps) transformer-based music captioning model zero-shot transfer-learning