Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large
  Language Models
Large Language Models (LLMs) demonstrate enhanced capabilities and
reliability by reasoning more, evolving from Chain-of-Thought prompting to
product-level solutions like OpenAI o1. Despite various efforts to improve LLM
reasoning, high-quality long-chain reasoning data and optimized training
pipelines still remain inadequately explored in vision-language tasks. In this
paper, we present Insight-V, an early effort to 1) scalably produce long and
robust reasoning data for complex multi-modal tasks, and 2) an effective
training pipeline to enhance the reasoning capabilities of multi-modal large
language models (MLLMs). Specifically, to create long and structured reasoning
data without human labor, we design a two-step pipeline with a progressive
strategy to generate sufficiently long and diverse reasoning paths and a
multi-granularity assessment method to ensure data quality. We observe that
directly supervising MLLMs with such long and complex reasoning data will not
yield ideal reasoning ability. To tackle this problem, we design a multi-agent
system consisting of a reasoning agent dedicated to performing long-chain
reasoning and a summary agent trained to judge and summarize reasoning results.
We further incorporate an iterative DPO algorithm to enhance the reasoning
agent's generation stability and quality. Based on the popular LLaVA-NeXT model
and our stronger base MLLM, we demonstrate significant performance gains across
challenging multi-modal benchmarks requiring visual reasoning. Benefiting from
our multi-agent system, Insight-V can also easily maintain or improve
performance on perception-focused multi-modal tasks.
673ff66bbf2e9c35b20570fe
Yuhao Dong
claimed_verified
2024-11-22T10:01:57.737Z
652965773a416e1f2173443b
https://cdn-avatars.huggingface.co/v1/production/uploads/652965773a416e1f2173443b/y9MB8YgHzbwCXAc4EI9T3.jpeg
Yuhao Dong
user
THUdyh
673ff66bbf2e9c35b20570ff
Zuyan Liu
claimed_verified
2024-11-22T10:01:56.110Z
64f001bfabd9fb1914398bd5
https://cdn-avatars.huggingface.co/v1/production/uploads/64f001bfabd9fb1914398bd5/9teH82hkBI4csIz_WQh5q.jpeg
liuzuyan
user
Zuyan
673ff66bbf2e9c35b2057100
Hai-Long Sun
admin_assigned
2024-11-22T13:15:29.257Z
64cb4e554726a3f8334b75f4
/avatars/f00328f69f88dc3f0bcf566e8316612d.svg
Hailong Sun
user
HailongSun
673ff66bbf2e9c35b2057101
Jingkang Yang
admin_assigned
2024-11-22T13:16:29.264Z
62b5777f593a2c49da69dc02
https://cdn-avatars.huggingface.co/v1/production/uploads/1658152070753-62b5777f593a2c49da69dc02.jpeg
Jingkang Yang
user
Jingkang
673ff66bbf2e9c35b2057102
Winston Hu
admin_assigned
2024-11-22T13:16:37.310Z
63673bb9d0ee6e2662be0ec1
/avatars/1b8976785d64bc4e3f7159ccdb7f06c5.svg
Qingqiao Hu
user
WinstonHu
673ff66bbf2e9c35b2057103
Yongming Rao
admin_assigned
2024-11-22T13:16:43.635Z
63e4865354f51ea342d45d78
/avatars/2e7eccc878751331ca8b282f53e38899.svg
Yongming Rao
user
raoyongming
673ff66bbf2e9c35b2057104
Ziwei Liu
admin_assigned
2024-11-22T13:16:50.055Z
62ab1ac1d48b4d8b048a3473
https://cdn-avatars.huggingface.co/v1/production/uploads/1656826685333-62ab1ac1d48b4d8b048a3473.png
Ziwei Liu
user
liuziwei7
Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large
  Language Models
Large Language Models (LLMs) demonstrate enhanced capabilities and
reliability by reasoning more, evolving from Chain-of-Thought prompting to
product-level solutions like OpenAI o1. Despite various efforts to improve LLM
reasoning, high-quality long-chain reasoning data and optimized training
pipelines still remain inadequately explored in vision-language tasks. In this
paper, we present Insight-V, an early effort to 1) scalably produce long and
robust reasoning data for complex multi-modal tasks, and 2) an effective
training pipeline to enhance the reasoning capabilities of multi-modal large
language models (MLLMs). Specifically, to create long and structured reasoning
data without human labor, we design a two-step pipeline with a progressive
strategy to generate sufficiently long and diverse reasoning paths and a
multi-granularity assessment method to ensure data quality. We observe that
directly supervising MLLMs with such long and complex reasoning data will not
yield ideal reasoning ability. To tackle this problem, we design a multi-agent
system consisting of a reasoning agent dedicated to performing long-chain
reasoning and a summary agent trained to judge and summarize reasoning results.
We further incorporate an iterative DPO algorithm to enhance the reasoning
agent's generation stability and quality. Based on the popular LLaVA-NeXT model
and our stronger base MLLM, we demonstrate significant performance gains across
challenging multi-modal benchmarks requiring visual reasoning. Benefiting from
our multi-agent system, Insight-V can also easily maintain or improve
performance on perception-focused multi-modal tasks.
Insight-V enhances multi-modal large language models through scalable reasoning data generation and a multi-agent system, achieving performance improvements in visual and perceptual tasks.
LLMs reasoning Chain-of-Thought OpenAI vision-language tasks multi-modal tasks long-chain reasoning data training pipeline multi-agent system reasoning agent summary agent DPO algorithm LLaVA-NeXT