title: SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual
    Reasoning Self-Improvement
summary: In this paper, we present an effective method to enhance visual reasoning
  with significantly fewer training samples, relying purely on self-improvement
  with no knowledge distillation. Our key insight is that the difficulty of
  training data during reinforcement fine-tuning (RFT) is critical. Appropriately
  challenging samples can substantially boost reasoning capabilities even when
  the dataset is small. Despite being intuitive, the main challenge remains in
  accurately quantifying sample difficulty to enable effective data filtering. To
  this end, we propose a novel way of repurposing Monte Carlo Tree Search (MCTS)
  to achieve that. Starting from our curated 70k open-source training samples, we
  introduce an MCTS-based selection method that quantifies sample difficulty
  based on the number of iterations required by the VLMs to solve each problem.
  This explicit step-by-step reasoning in MCTS enforces the model to think longer
  and better identifies samples that are genuinely challenging. We filter and
  retain 11k samples to perform RFT on Qwen2.5-VL-7B-Instruct, resulting in our
  final model, ThinkLite-VL. Evaluation results on eight benchmarks show that
  ThinkLite-VL improves the average performance of Qwen2.5-VL-7B-Instruct by 7%,
  using only 11k training samples with no knowledge distillation. This
  significantly outperforms all existing 7B-level reasoning VLMs, and our fairly
  comparable baselines that use classic selection methods such as accuracy-based
  filtering. Notably, on MathVista, ThinkLite-VL-7B achieves the SoTA accuracy of
  75.1, surpassing Qwen2.5-VL-72B, GPT-4o, and O1. Our code, data, and model are
  available at https://github.com/si0wang/ThinkLite-VL.
paper_authors._id: 67f88bbaf1410163f7c3b68a
paper_authors.name: Xiyao Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-11T08:32:46.460Z
paper_authors._id: 655fed9fdef5905d38b84af3
paper_authors.avatarUrl: /avatars/2cda4182dfd11a1e94743639e62328ea.svg
paper_authors.fullname: Xiyao Wang
paper_authors.type: user
paper_authors.user: russwang
paper_authors._id: 67f88bbaf1410163f7c3b68b
paper_authors.name: Zhengyuan Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-11T08:32:53.708Z
paper_authors._id: 630713411801ecc7d2592a7c
paper_authors.avatarUrl: /avatars/fb36f69f03421c3a2a7f72ba0858fa60.svg
paper_authors.fullname: Zhengyuan Yang
paper_authors.type: user
paper_authors.user: zyang39
paper_authors._id: 67f88bbaf1410163f7c3b68c
paper_authors.name: Chao Feng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-13T19:25:33.292Z
paper_authors._id: 645ab0b7c266796265baefa4
paper_authors.avatarUrl: /avatars/bdac661996b63c4b2a56881707afa01f.svg
paper_authors.fullname: Chao Feng
paper_authors.type: user
paper_authors.user: chfeng
paper_authors._id: 67f88bbaf1410163f7c3b68d
paper_authors.name: Hongjin Lu
paper_authors._id: 67f88bbaf1410163f7c3b68e
paper_authors.name: Linjie Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-11T08:33:27.401Z
paper_authors._id: 63db16fff03c3d71ef397206
paper_authors.avatarUrl: /avatars/bfb7e0d730b7d03302799d5d2828d97d.svg
paper_authors.fullname: Linjie Li
paper_authors.type: user
paper_authors.user: linjieli222
paper_authors._id: 67f88bbaf1410163f7c3b68f
paper_authors.name: Chung-Ching Lin
paper_authors._id: 67f88bbaf1410163f7c3b690
paper_authors.name: Kevin Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-11T08:33:58.928Z
paper_authors._id: 6298fd95b58e71e2ac9f3ad8
paper_authors.avatarUrl: /avatars/7d34644d537bc5c17cf1e4ce4095355c.svg
paper_authors.fullname: Kevin Lin
paper_authors.type: user
paper_authors.user: kevinlin311tw
paper_authors._id: 67f88bbaf1410163f7c3b691
paper_authors.name: Furong Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-11T08:34:07.020Z
paper_authors._id: 64cbc3e2a257a3212c00a115
paper_authors.avatarUrl: /avatars/836e61be4aeda2080ddf2db9f2626cc6.svg
paper_authors.fullname: Furong Huang Lab at UMD
paper_authors.type: user
paper_authors.user: furongh-lab
paper_authors._id: 67f88bbaf1410163f7c3b692
paper_authors.name: Lijuan Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-11T08:34:25.295Z
paper_authors._id: 6413521d4e5305c14f22e110
paper_authors.avatarUrl: /avatars/a6f8d0573e678f79bc3c0b7897b818ce.svg
paper_authors.fullname: Lijuan Wang
paper_authors.type: user
paper_authors.user: Lijuan
paper_title: SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual
    Reasoning Self-Improvement
paper_summary: In this paper, we present an effective method to enhance visual reasoning
  with significantly fewer training samples, relying purely on self-improvement
  with no knowledge distillation. Our key insight is that the difficulty of
  training data during reinforcement fine-tuning (RFT) is critical. Appropriately
  challenging samples can substantially boost reasoning capabilities even when
  the dataset is small. Despite being intuitive, the main challenge remains in
  accurately quantifying sample difficulty to enable effective data filtering. To
  this end, we propose a novel way of repurposing Monte Carlo Tree Search (MCTS)
  to achieve that. Starting from our curated 70k open-source training samples, we
  introduce an MCTS-based selection method that quantifies sample difficulty
  based on the number of iterations required by the VLMs to solve each problem.
  This explicit step-by-step reasoning in MCTS enforces the model to think longer
  and better identifies samples that are genuinely challenging. We filter and
  retain 11k samples to perform RFT on Qwen2.5-VL-7B-Instruct, resulting in our
  final model, ThinkLite-VL. Evaluation results on eight benchmarks show that
  ThinkLite-VL improves the average performance of Qwen2.5-VL-7B-Instruct by 7%,
  using only 11k training samples with no knowledge distillation. This
  significantly outperforms all existing 7B-level reasoning VLMs, and our fairly
  comparable baselines that use classic selection methods such as accuracy-based
  filtering. Notably, on MathVista, ThinkLite-VL-7B achieves the SoTA accuracy of
  75.1, surpassing Qwen2.5-VL-72B, GPT-4o, and O1. Our code, data, and model are
  available at https://github.com/si0wang/ThinkLite-VL.
paper_ai_summary: A method using Monte Carlo Tree Search for selecting challenging training samples enhances visual reasoning capabilities in a smaller dataset compared to existing models.
paper_ai_keywords: reinforcement fine-tuning Monte Carlo Tree Search sample difficulty VLMs visual reasoning Qwen2.5-VL-7B-Instruct ThinkLite-VL MathVista SoTA accuracy