title: Towards Self-Improving Systematic Cognition for Next-Generation
    Foundation MLLMs
summary: Despite their impressive capabilities, Multimodal Large Language Models
  (MLLMs) face challenges with fine-grained perception and complex reasoning.
  Prevalent multimodal pre-training approaches focus on enhancing perception by
  training on high-quality image captions due to the extremely high cost of
  collecting chain-of-thought (CoT) reasoning data for improving reasoning. While
  leveraging advanced MLLMs for caption generation enhances scalability, the
  outputs often lack comprehensiveness and accuracy. In this paper, we introduce
  Self-Improving cognition (SIcog), a self-learning framework designed to
  construct next-generation foundation MLLMs by enhancing their systematic
  cognitive capabilities through multimodal pre-training with self-generated
  data. Specifically, we propose Chain-of-Description, an approach that improves
  an MLLM's systematic perception by enabling step-by-step visual understanding,
  ensuring greater comprehensiveness and accuracy. Additionally, we adopt a
  structured CoT reasoning technique to enable MLLMs to integrate in-depth
  multimodal reasoning. To construct a next-generation foundation MLLM with
  self-improved cognition, SIcog first equips an MLLM with systematic perception
  and reasoning abilities using minimal external annotations. The enhanced models
  then generate detailed captions and CoT reasoning data, which are further
  curated through self-consistency. This curated data is ultimately used for
  multimodal pre-training to develop next-generation foundation models. Extensive
  experiments on both low- and high-resolution MLLMs across diverse benchmarks
  demonstrate that, with merely 213K self-generated pre-training samples, SIcog
  produces next-generation foundation MLLMs with significantly improved
  cognition, achieving benchmark-leading performance compared to prevalent
  pre-training approaches.
paper_authors._id: 67da52cd11b6db69208527b9
paper_authors.name: Xiaoying Zhang
paper_authors._id: 67da52cd11b6db69208527ba
paper_authors.name: Da Peng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-19T09:44:20.846Z
paper_authors._id: 665b276ea132595fafc4a431
paper_authors.avatarUrl: /avatars/f238582a506bc24a822360b4d4cbd625.svg
paper_authors.fullname: PengDa
paper_authors.type: user
paper_authors.user: PengDa02
paper_authors._id: 67da52cd11b6db69208527bb
paper_authors.name: Yipeng Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-19T09:44:16.169Z
paper_authors._id: 666e91b1623133f1ce35acc5
paper_authors.avatarUrl: /avatars/cc78520e6cfb83817c1d0c1ac867ebdd.svg
paper_authors.fullname: YipengZhang
paper_authors.type: user
paper_authors.user: YipengZhang
paper_authors._id: 67da52cd11b6db69208527bc
paper_authors.name: Zonghao Guo
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-03-19T05:14:57.514Z
paper_authors._id: 6491af36c1741666238f3bff
paper_authors.avatarUrl: /avatars/0ee7d2ec1566e2cc5e8f144140e17f00.svg
paper_authors.fullname: Zonghao Guo
paper_authors.type: user
paper_authors.user: guozonghao96
paper_authors._id: 67da52cd11b6db69208527bd
paper_authors.name: Chengyue Wu
paper_authors._id: 67da52cd11b6db69208527be
paper_authors.name: Chi Chen
paper_authors._id: 67da52cd11b6db69208527bf
paper_authors.name: Wei Ke
paper_authors._id: 67da52cd11b6db69208527c0
paper_authors.name: Helen Meng
paper_authors._id: 67da52cd11b6db69208527c1
paper_authors.name: Maosong Sun
paper_title: Towards Self-Improving Systematic Cognition for Next-Generation
    Foundation MLLMs
paper_summary: Despite their impressive capabilities, Multimodal Large Language Models
  (MLLMs) face challenges with fine-grained perception and complex reasoning.
  Prevalent multimodal pre-training approaches focus on enhancing perception by
  training on high-quality image captions due to the extremely high cost of
  collecting chain-of-thought (CoT) reasoning data for improving reasoning. While
  leveraging advanced MLLMs for caption generation enhances scalability, the
  outputs often lack comprehensiveness and accuracy. In this paper, we introduce
  Self-Improving cognition (SIcog), a self-learning framework designed to
  construct next-generation foundation MLLMs by enhancing their systematic
  cognitive capabilities through multimodal pre-training with self-generated
  data. Specifically, we propose Chain-of-Description, an approach that improves
  an MLLM's systematic perception by enabling step-by-step visual understanding,
  ensuring greater comprehensiveness and accuracy. Additionally, we adopt a
  structured CoT reasoning technique to enable MLLMs to integrate in-depth
  multimodal reasoning. To construct a next-generation foundation MLLM with
  self-improved cognition, SIcog first equips an MLLM with systematic perception
  and reasoning abilities using minimal external annotations. The enhanced models
  then generate detailed captions and CoT reasoning data, which are further
  curated through self-consistency. This curated data is ultimately used for
  multimodal pre-training to develop next-generation foundation models. Extensive
  experiments on both low- and high-resolution MLLMs across diverse benchmarks
  demonstrate that, with merely 213K self-generated pre-training samples, SIcog
  produces next-generation foundation MLLMs with significantly improved
  cognition, achieving benchmark-leading performance compared to prevalent
  pre-training approaches.
paper_ai_summary: SIcog enhances MLLMs through self-generated data and structured reasoning to improve systematic perception and reasoning, achieving superior performance with limited annotations.
paper_ai_keywords: Multimodal Large Language Models (MLLMs) fine-grained perception complex reasoning chain-of-thought (CoT) reasoning self-improving cognition (SIcog) Chain-of-Description structured CoT reasoning multimodal pre-training self-generated data self-consistency