title: Brain2Music: Reconstructing Music from Human Brain Activity
summary: The process of reconstructing experiences from human brain activity offers a
  unique lens into how the brain interprets and represents the world. In this
  paper, we introduce a method for reconstructing music from brain activity,
  captured using functional magnetic resonance imaging (fMRI). Our approach uses
  either music retrieval or the MusicLM music generation model conditioned on
  embeddings derived from fMRI data. The generated music resembles the musical
  stimuli that human subjects experienced, with respect to semantic properties
  like genre, instrumentation, and mood. We investigate the relationship between
  different components of MusicLM and brain activity through a voxel-wise
  encoding modeling analysis. Furthermore, we discuss which brain regions
  represent information derived from purely textual descriptions of music
  stimuli. We provide supplementary material including examples of the
  reconstructed music at https://google-research.github.io/seanet/brain2music
paper_authors._id: 64ba0a0e594d62250b02198c
paper_authors.name: Timo I. Denk
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-21T08:40:35.916Z
paper_authors._id: 64897be5308f19bc7c433891
paper_authors.avatarUrl: /avatars/b716cc5b110942a02b1e3aa001c86125.svg
paper_authors.fullname: Timo Denk
paper_authors.type: user
paper_authors.user: Simsso
paper_authors._id: 64ba0a0e594d62250b02198d
paper_authors.name: Yu Takagi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-21T07:47:24.405Z
paper_authors._id: 63052ea153b3899eeb6142c3
paper_authors.avatarUrl: /avatars/c42eda9776ce9c7f3510aab91754c0bd.svg
paper_authors.fullname: Yu Takagi
paper_authors.type: user
paper_authors.user: yu-takagi
paper_authors._id: 64ba0a0e594d62250b02198e
paper_authors.name: Takuya Matsuyama
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-21T07:47:42.033Z
paper_authors._id: 64b8ab6fe0bd7b05a844a18f
paper_authors.avatarUrl: /avatars/68e0af09c7db6163a1542e744af78a2f.svg
paper_authors.fullname: Takuya Matsuyama
paper_authors.type: user
paper_authors.user: suyamat
paper_authors._id: 64ba0a0e594d62250b02198f
paper_authors.name: Andrea Agostinelli
paper_authors._id: 64ba0a0e594d62250b021990
paper_authors.name: Tomoya Nakai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-21T07:48:53.544Z
paper_authors._id: 64aa6bace04e7f9224665caa
paper_authors.avatarUrl: /avatars/5a9cd4e09253bc1257221cc182d246f8.svg
paper_authors.fullname: Tomoya Nakai
paper_authors.type: user
paper_authors.user: TN333
paper_authors._id: 64ba0a0e594d62250b021991
paper_authors.name: Christian Frank
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-21T19:24:56.081Z
paper_authors._id: 64bab1dd140491ca9f0eced3
paper_authors.avatarUrl: /avatars/b9fef1e01c064fbee38aa110de8f3bf5.svg
paper_authors.fullname: Christian Frank
paper_authors.type: user
paper_authors.user: chfrank
paper_authors._id: 64ba0a0e594d62250b021992
paper_authors.name: Shinji Nishimoto
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-22T08:42:57.524Z
paper_authors._id: 62fe90d5425f04f019605a57
paper_authors.avatarUrl: /avatars/25687065605961b5d844394296810c35.svg
paper_authors.fullname: Shinji Nishimoto
paper_authors.type: user
paper_authors.user: shinjinishimoto
paper_title: Brain2Music: Reconstructing Music from Human Brain Activity
paper_summary: The process of reconstructing experiences from human brain activity offers a
  unique lens into how the brain interprets and represents the world. In this
  paper, we introduce a method for reconstructing music from brain activity,
  captured using functional magnetic resonance imaging (fMRI). Our approach uses
  either music retrieval or the MusicLM music generation model conditioned on
  embeddings derived from fMRI data. The generated music resembles the musical
  stimuli that human subjects experienced, with respect to semantic properties
  like genre, instrumentation, and mood. We investigate the relationship between
  different components of MusicLM and brain activity through a voxel-wise
  encoding modeling analysis. Furthermore, we discuss which brain regions
  represent information derived from purely textual descriptions of music
  stimuli. We provide supplementary material including examples of the
  reconstructed music at https://google-research.github.io/seanet/brain2music
paper_ai_keywords: functional magnetic resonance imaging (fMRI) MusicLM music retrieval music generation model voxel-wise encoding modeling brain regions