Music Arena: Live Evaluation for Text-to-Music
We present Music Arena, an open platform for scalable human preference
evaluation of text-to-music (TTM) models. Soliciting human preferences via
listening studies is the gold standard for evaluation in TTM, but these studies
are expensive to conduct and difficult to compare, as study protocols may
differ across systems. Moreover, human preferences might help researchers align
their TTM systems or improve automatic evaluation metrics, but an open and
renewable source of preferences does not currently exist. We aim to fill these
gaps by offering *live* evaluation for TTM. In Music Arena, real-world users
input text prompts of their choosing and compare outputs from two TTM systems,
and their preferences are used to compile a leaderboard. While Music Arena
follows recent evaluation trends in other AI domains, we also design it with
key features tailored to music: an LLM-based routing system to navigate the
heterogeneous type signatures of TTM systems, and the collection of *detailed*
preferences including listening data and natural language feedback. We also
propose a rolling data release policy with user privacy guarantees, providing a
renewable source of preference data and increasing platform transparency.
Through its standardized evaluation protocol, transparent data access policies,
and music-specific features, Music Arena not only addresses key challenges in
the TTM ecosystem but also demonstrates how live evaluation can be thoughtfully
adapted to unique characteristics of specific AI domains.
  Music Arena is available at: https://music-arena.org
68883026af872d625c10c5e1
Yonghyun Kim
claimed_verified
2025-07-30T09:05:19.018Z
67dc6c6e8fc6577e1851b36e
https://cdn-avatars.huggingface.co/v1/production/uploads/67dc6c6e8fc6577e1851b36e/Sb0-erpwIC4Qutty5KEln.jpeg
Yonghyun Kim
user
yonghyunk1m
68883026af872d625c10c5e2
Wayne Chi
68883026af872d625c10c5e3
Anastasios N. Angelopoulos
68883026af872d625c10c5e4
Wei-Lin Chiang
68883026af872d625c10c5e5
Koichi Saito
68883026af872d625c10c5e6
Shinji Watanabe
68883026af872d625c10c5e7
Yuki Mitsufuji
68883026af872d625c10c5e8
Chris Donahue
claimed_verified
2025-07-29T02:34:38.516Z
63b1b5f034b91b8272aac6c8
/avatars/155351aa15967b5346144c03fc87d1ca.svg
Chris Donahue
user
chrisdonahue
Music Arena: Live Evaluation for Text-to-Music
We present Music Arena, an open platform for scalable human preference
evaluation of text-to-music (TTM) models. Soliciting human preferences via
listening studies is the gold standard for evaluation in TTM, but these studies
are expensive to conduct and difficult to compare, as study protocols may
differ across systems. Moreover, human preferences might help researchers align
their TTM systems or improve automatic evaluation metrics, but an open and
renewable source of preferences does not currently exist. We aim to fill these
gaps by offering *live* evaluation for TTM. In Music Arena, real-world users
input text prompts of their choosing and compare outputs from two TTM systems,
and their preferences are used to compile a leaderboard. While Music Arena
follows recent evaluation trends in other AI domains, we also design it with
key features tailored to music: an LLM-based routing system to navigate the
heterogeneous type signatures of TTM systems, and the collection of *detailed*
preferences including listening data and natural language feedback. We also
propose a rolling data release policy with user privacy guarantees, providing a
renewable source of preference data and increasing platform transparency.
Through its standardized evaluation protocol, transparent data access policies,
and music-specific features, Music Arena not only addresses key challenges in
the TTM ecosystem but also demonstrates how live evaluation can be thoughtfully
adapted to unique characteristics of specific AI domains.
  Music Arena is available at: https://music-arena.org
Music Arena is an open platform for real-time human preference evaluation of text-to-music models, featuring a standardized protocol, detailed feedback, and privacy guarantees.
text-to-music LLM-based routing system listening data natural language feedback rolling data release policy user privacy guarantees