MusiXQA: Advancing Visual Music Understanding in Multimodal Large
  Language Models
Multimodal Large Language Models (MLLMs) have achieved remarkable visual
reasoning abilities in natural images, text-rich documents, and graphic
designs. However, their ability to interpret music sheets remains
underexplored. To bridge this gap, we introduce MusiXQA, the first
comprehensive dataset for evaluating and advancing MLLMs in music sheet
understanding. MusiXQA features high-quality synthetic music sheets generated
via MusiXTeX, with structured annotations covering note pitch and duration,
chords, clefs, key/time signatures, and text, enabling diverse visual QA tasks.
Through extensive evaluations, we reveal significant limitations of current
state-of-the-art MLLMs in this domain. Beyond benchmarking, we developed
Phi-3-MusiX, an MLLM fine-tuned on our dataset, achieving significant
performance gains over GPT-based methods. The proposed dataset and model
establish a foundation for future advances in MLLMs for music sheet
understanding. Code, data, and model will be released upon acceptance.
6865807e554cb97d1ed7282c
Jian Chen
claimed_verified
2025-07-03T07:17:10.525Z
64593836c16ecb4815e082ce
https://cdn-avatars.huggingface.co/v1/production/uploads/64593836c16ecb4815e082ce/T9WM0Ly6ADi3a9JUSIsSV.png
Jian Chen
user
puar-playground
6865807e554cb97d1ed7282d
Wenye Ma
claimed_verified
2025-07-03T07:17:07.793Z
67c422d865526859fbdd2596
/avatars/de1fb6c53ac770c4ae0dfeb03aac0d04.svg
Wenye Ma
user
chestnutma
6865807e554cb97d1ed7282e
Penghang Liu
6865807e554cb97d1ed7282f
Wei Wang
6865807e554cb97d1ed72830
Tengwei Song
claimed_verified
2025-07-04T07:50:35.907Z
673305196c671fb849c59564
/avatars/bc1091360dc60a65a5b752b01d45e72e.svg
Tengwei Song
user
tewaye
6865807e554cb97d1ed72831
Ming Li
6865807e554cb97d1ed72832
Chenguang Wang
6865807e554cb97d1ed72833
Ruiyi Zhang
6865807e554cb97d1ed72834
Changyou Chen
MusiXQA: Advancing Visual Music Understanding in Multimodal Large
  Language Models
Multimodal Large Language Models (MLLMs) have achieved remarkable visual
reasoning abilities in natural images, text-rich documents, and graphic
designs. However, their ability to interpret music sheets remains
underexplored. To bridge this gap, we introduce MusiXQA, the first
comprehensive dataset for evaluating and advancing MLLMs in music sheet
understanding. MusiXQA features high-quality synthetic music sheets generated
via MusiXTeX, with structured annotations covering note pitch and duration,
chords, clefs, key/time signatures, and text, enabling diverse visual QA tasks.
Through extensive evaluations, we reveal significant limitations of current
state-of-the-art MLLMs in this domain. Beyond benchmarking, we developed
Phi-3-MusiX, an MLLM fine-tuned on our dataset, achieving significant
performance gains over GPT-based methods. The proposed dataset and model
establish a foundation for future advances in MLLMs for music sheet
understanding. Code, data, and model will be released upon acceptance.
MusiXQA, a new dataset for evaluating MLLMs on music sheet understanding, reveals limitations and enables the development of Phi-3-MusiX, an improved MLLM for this task.
Multimodal Large Language Models MLLMs MusiXQA MusiXTeX visual QA tasks Phi-3-MusiX GPT-based methods