Art2Mus: Bridging Visual Arts and Music through Cross-Modal Generation
<|reference_start|>Art2Mus: Bridging Visual Arts and Music through Cross-Modal Generation: Artificial Intelligence and generative models have revolutionized music creation, with many models leveraging textual or visual prompts for guidance. However, existing image-to-music models are limited to simple images, lacking the capability to generate music from complex digitized artworks. To address this gap, we introduce $\mathcal{A}\textit{rt2}\mathcal{M}\textit{us}$, a novel model designed to create music from digitized artworks or text inputs. $\mathcal{A}\textit{rt2}\mathcal{M}\textit{us}$ extends the AudioLDM~2 architecture, a text-to-audio model, and employs our newly curated datasets, created via ImageBind, which pair digitized artworks with music. Experimental results demonstrate that $\mathcal{A}\textit{rt2}\mathcal{M}\textit{us}$ can generate music that resonates with the input stimuli. These findings suggest promising applications in multimedia art, interactive installations, and AI-driven creative tools.<|reference_end|>
@article{rinaldi2024art2mus:,
    title={Art2Mus: Bridging Visual Arts and Music through Cross-Modal Generation},
    author={Ivan Rinaldi, Nicola Fanelli, Giovanna Castellano, Gennaro Vessio},
    journal={arXiv preprint arXiv:2410.04906},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.04906},
    primaryClass={cs.MM cs.CV cs.SD eess.AS}
}