MEXA: Towards General Multimodal Reasoning with Dynamic Multi-Expert
  Aggregation
Combining pre-trained expert models offers substantial potential for scalable
multimodal reasoning, but building a unified framework remains challenging due
to the increasing diversity of input modalities and task complexity. For
instance, medical diagnosis requires precise reasoning over structured clinical
tables, while financial forecasting depends on interpreting plot-based data to
make informed predictions. To tackle this challenge, we introduce MEXA, a
training-free framework that performs modality- and task-aware aggregation of
multiple expert models to enable effective multimodal reasoning across diverse
and distinct domains. MEXA dynamically selects expert models based on the input
modality and the task-specific reasoning demands (i.e., skills). Each expert
model, specialized in a modality task pair, generates interpretable textual
reasoning outputs. MEXA then aggregates and reasons over these outputs using a
Large Reasoning Model (LRM) to produce the final answer. This modular design
allows flexible and transparent multimodal reasoning across diverse domains
without additional training overhead. We extensively evaluate our approach on
diverse multimodal benchmarks, including Video Reasoning, Audio Reasoning, 3D
Understanding, and Medical QA. MEXA consistently delivers performance
improvements over strong multimodal baselines, highlighting the effectiveness
and broad applicability of our expert-driven selection and aggregation in
diverse multimodal reasoning tasks.
68597ef30e4ad7e2197583d0
Shoubin Yu
68597ef30e4ad7e2197583d1
Yue Zhang
68597ef30e4ad7e2197583d2
Ziyang Wang
68597ef30e4ad7e2197583d3
Jaehong Yoon
68597ef30e4ad7e2197583d4
Mohit Bansal
MEXA: Towards General Multimodal Reasoning with Dynamic Multi-Expert
  Aggregation
Combining pre-trained expert models offers substantial potential for scalable
multimodal reasoning, but building a unified framework remains challenging due
to the increasing diversity of input modalities and task complexity. For
instance, medical diagnosis requires precise reasoning over structured clinical
tables, while financial forecasting depends on interpreting plot-based data to
make informed predictions. To tackle this challenge, we introduce MEXA, a
training-free framework that performs modality- and task-aware aggregation of
multiple expert models to enable effective multimodal reasoning across diverse
and distinct domains. MEXA dynamically selects expert models based on the input
modality and the task-specific reasoning demands (i.e., skills). Each expert
model, specialized in a modality task pair, generates interpretable textual
reasoning outputs. MEXA then aggregates and reasons over these outputs using a
Large Reasoning Model (LRM) to produce the final answer. This modular design
allows flexible and transparent multimodal reasoning across diverse domains
without additional training overhead. We extensively evaluate our approach on
diverse multimodal benchmarks, including Video Reasoning, Audio Reasoning, 3D
Understanding, and Medical QA. MEXA consistently delivers performance
improvements over strong multimodal baselines, highlighting the effectiveness
and broad applicability of our expert-driven selection and aggregation in
diverse multimodal reasoning tasks.
MEXA is a training-free framework that aggregates outputs from specialized expert models using a Large Reasoning Model for effective multimodal reasoning across various domains.
pre-trained expert models multimodal reasoning input modalities task-specific reasoning modality- and task-aware aggregation Large Reasoning Model Video Reasoning Audio Reasoning 3D Understanding Medical QA