title: BBA: Bi-Modal Behavioral Alignment for Reasoning with Large
    Vision-Language Models
summary: Multimodal reasoning stands as a pivotal capability for large vision-language
  models (LVLMs). The integration with Domain-Specific Languages (DSL), offering
  precise visual representations, equips these models with the opportunity to
  execute more accurate reasoning in complex and professional domains. However,
  the vanilla Chain-of-Thought (CoT) prompting method faces challenges in
  effectively leveraging the unique strengths of visual and DSL representations,
  primarily due to their differing reasoning mechanisms. Additionally, it often
  falls short in addressing critical steps in multi-step reasoning tasks. To
  mitigate these challenges, we introduce the Bi-Modal
  Behavioral Alignment (BBA) prompting method, designed
  to maximize the potential of DSL in augmenting complex multi-modal reasoning
  tasks. This method initiates by guiding LVLMs to create separate reasoning
  chains for visual and DSL representations. Subsequently, it aligns these chains
  by addressing any inconsistencies, thus achieving a cohesive integration of
  behaviors from different modalities. Our experiments demonstrate that BBA
  substantially improves the performance of GPT-4V(ision) on geometry problem
  solving (28.34% to 34.22%), chess positional advantage prediction
  (42.08% to 46.99%) and molecular property prediction (77.47% to
  83.52%).
paper_authors._id: 65d6c473b20efa5a5f390a90
paper_authors.name: Xueliang Zhao
paper_authors._id: 65d6c473b20efa5a5f390a91
paper_authors.name: Xinting Huang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-05-14T13:51:51.164Z
paper_authors._id: 63c227a6c58fcfeac18dec07
paper_authors.avatarUrl: /avatars/47df92c937cfaf8d726c7e28cf352eb5.svg
paper_authors.fullname: tim huang
paper_authors.type: user
paper_authors.user: timmyhhh
paper_authors._id: 65d6c473b20efa5a5f390a92
paper_authors.name: Tingchen Fu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-22T07:20:10.407Z
paper_authors._id: 640ad17a1ee054d66a74783e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/640ad17a1ee054d66a74783e/u0PjIkyC-9HkGzEyUQ7JN.jpeg
paper_authors.fullname: Tingchen Fu
paper_authors.type: user
paper_authors.user: TingchenFu
paper_authors._id: 65d6c473b20efa5a5f390a93
paper_authors.name: Qintong Li
paper_authors._id: 65d6c473b20efa5a5f390a94
paper_authors.name: Shansan Gong
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-24T07:16:38.473Z
paper_authors._id: 628c83d186fc004b14e1ed48
paper_authors.avatarUrl: /avatars/05ff943a9b89b5f67c5bc254bf45b8f5.svg
paper_authors.fullname: Shansan Gong
paper_authors.type: user
paper_authors.user: Sansa
paper_authors._id: 65d6c473b20efa5a5f390a95
paper_authors.name: Lemao Liu
paper_authors._id: 65d6c473b20efa5a5f390a96
paper_authors.name: Wei Bi
paper_authors._id: 65d6c473b20efa5a5f390a97
paper_authors.name: Lingpeng Kong
paper_title: BBA: Bi-Modal Behavioral Alignment for Reasoning with Large
    Vision-Language Models
paper_summary: Multimodal reasoning stands as a pivotal capability for large vision-language
  models (LVLMs). The integration with Domain-Specific Languages (DSL), offering
  precise visual representations, equips these models with the opportunity to
  execute more accurate reasoning in complex and professional domains. However,
  the vanilla Chain-of-Thought (CoT) prompting method faces challenges in
  effectively leveraging the unique strengths of visual and DSL representations,
  primarily due to their differing reasoning mechanisms. Additionally, it often
  falls short in addressing critical steps in multi-step reasoning tasks. To
  mitigate these challenges, we introduce the Bi-Modal
  Behavioral Alignment (BBA) prompting method, designed
  to maximize the potential of DSL in augmenting complex multi-modal reasoning
  tasks. This method initiates by guiding LVLMs to create separate reasoning
  chains for visual and DSL representations. Subsequently, it aligns these chains
  by addressing any inconsistencies, thus achieving a cohesive integration of
  behaviors from different modalities. Our experiments demonstrate that BBA
  substantially improves the performance of GPT-4V(ision) on geometry problem
  solving (28.34% to 34.22%), chess positional advantage prediction
  (42.08% to 46.99%) and molecular property prediction (77.47% to
  83.52%).
paper_ai_summary: The BBA prompting method enhances multimodal reasoning in vision-language models by integrating visual and DSL representations through separate reasoning chains and behavioral alignment, improving performance across various complex tasks.
paper_ai_keywords: vision-language models multimodal reasoning Domain-Specific Languages Chain-of-Thought BBA prompting method reasoning chains behavioral alignment GPT-4V geometry problem solving chess positional advantage prediction molecular property prediction