title: A Multimodal Symphony: Integrating Taste and Sound through Generative AI
summary: In recent decades, neuroscientific and psychological research has traced
  direct relationships between taste and auditory perceptions. This article
  explores multimodal generative models capable of converting taste information
  into music, building on this foundational research. We provide a brief review
  of the state of the art in this field, highlighting key findings and
  methodologies. We present an experiment in which a fine-tuned version of a
  generative music model (MusicGEN) is used to generate music based on detailed
  taste descriptions provided for each musical piece. The results are promising:
  according the participants' (n=111) evaluation, the fine-tuned model produces
  music that more coherently reflects the input taste descriptions compared to
  the non-fine-tuned model. This study represents a significant step towards
  understanding and developing embodied interactions between AI, sound, and
  taste, opening new possibilities in the field of generative AI. We release our
  dataset, code and pre-trained model at: https://osf.io/xs5jy/.
paper_authors._id: 67c8522f8cc8a6393971a16c
paper_authors.name: Matteo Spanio
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-05T15:46:46.617Z
paper_authors._id: 65b00f6bc9a5a7680f531c1d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65b00f6bc9a5a7680f531c1d/z4hzKR5tgdkft6c3Wxt9Y.jpeg
paper_authors.fullname: Matteo Spanio
paper_authors.type: user
paper_authors.user: matteospanio
paper_authors._id: 67c8522f8cc8a6393971a16d
paper_authors.name: Massimiliano Zampini
paper_authors._id: 67c8522f8cc8a6393971a16e
paper_authors.name: Antonio Rodà
paper_authors._id: 67c8522f8cc8a6393971a16f
paper_authors.name: Franco Pierucci
paper_title: A Multimodal Symphony: Integrating Taste and Sound through Generative AI
paper_summary: In recent decades, neuroscientific and psychological research has traced
  direct relationships between taste and auditory perceptions. This article
  explores multimodal generative models capable of converting taste information
  into music, building on this foundational research. We provide a brief review
  of the state of the art in this field, highlighting key findings and
  methodologies. We present an experiment in which a fine-tuned version of a
  generative music model (MusicGEN) is used to generate music based on detailed
  taste descriptions provided for each musical piece. The results are promising:
  according the participants' (n=111) evaluation, the fine-tuned model produces
  music that more coherently reflects the input taste descriptions compared to
  the non-fine-tuned model. This study represents a significant step towards
  understanding and developing embodied interactions between AI, sound, and
  taste, opening new possibilities in the field of generative AI. We release our
  dataset, code and pre-trained model at: https://osf.io/xs5jy/.
paper_ai_keywords: generative music model MusicGEN fine-tuned taste descriptions embodied interactions