SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual
  Reasoning Self-Improvement
In this paper, we present an effective method to enhance visual reasoning
with significantly fewer training samples, relying purely on self-improvement
with no knowledge distillation. Our key insight is that the difficulty of
training data during reinforcement fine-tuning (RFT) is critical. Appropriately
challenging samples can substantially boost reasoning capabilities even when
the dataset is small. Despite being intuitive, the main challenge remains in
accurately quantifying sample difficulty to enable effective data filtering. To
this end, we propose a novel way of repurposing Monte Carlo Tree Search (MCTS)
to achieve that. Starting from our curated 70k open-source training samples, we
introduce an MCTS-based selection method that quantifies sample difficulty
based on the number of iterations required by the VLMs to solve each problem.
This explicit step-by-step reasoning in MCTS enforces the model to think longer
and better identifies samples that are genuinely challenging. We filter and
retain 11k samples to perform RFT on Qwen2.5-VL-7B-Instruct, resulting in our
final model, ThinkLite-VL. Evaluation results on eight benchmarks show that
ThinkLite-VL improves the average performance of Qwen2.5-VL-7B-Instruct by 7%,
using only 11k training samples with no knowledge distillation. This
significantly outperforms all existing 7B-level reasoning VLMs, and our fairly
comparable baselines that use classic selection methods such as accuracy-based
filtering. Notably, on MathVista, ThinkLite-VL-7B achieves the SoTA accuracy of
75.1, surpassing Qwen2.5-VL-72B, GPT-4o, and O1. Our code, data, and model are
available at https://github.com/si0wang/ThinkLite-VL.
67f88bbaf1410163f7c3b68a
Xiyao Wang
admin_assigned
2025-04-11T08:32:46.460Z
655fed9fdef5905d38b84af3
/avatars/2cda4182dfd11a1e94743639e62328ea.svg
Xiyao Wang
user
russwang
67f88bbaf1410163f7c3b68b
Zhengyuan Yang
admin_assigned
2025-04-11T08:32:53.708Z
630713411801ecc7d2592a7c
/avatars/fb36f69f03421c3a2a7f72ba0858fa60.svg
Zhengyuan Yang
user
zyang39
67f88bbaf1410163f7c3b68c
Chao Feng
claimed_verified
2025-04-13T19:25:33.292Z
645ab0b7c266796265baefa4
/avatars/bdac661996b63c4b2a56881707afa01f.svg
Chao Feng
user
chfeng
67f88bbaf1410163f7c3b68d
Hongjin Lu
67f88bbaf1410163f7c3b68e
Linjie Li
admin_assigned
2025-04-11T08:33:27.401Z
63db16fff03c3d71ef397206
/avatars/bfb7e0d730b7d03302799d5d2828d97d.svg
Linjie Li
user
linjieli222
67f88bbaf1410163f7c3b68f
Chung-Ching Lin
claimed_verified
2025-07-23T08:38:09.178Z
6669f406d21cf7d39b1d98ba
/avatars/5ec270cb8bcb68786379d8a8bc411aaa.svg
CC Lin
user
cclin10
67f88bbaf1410163f7c3b690
Kevin Lin
admin_assigned
2025-04-11T08:33:58.928Z
6298fd95b58e71e2ac9f3ad8
/avatars/7d34644d537bc5c17cf1e4ce4095355c.svg
Kevin Lin
user
kevinlin311tw
67f88bbaf1410163f7c3b691
Furong Huang
admin_assigned
2025-04-11T08:34:07.020Z
64cbc3e2a257a3212c00a115
/avatars/836e61be4aeda2080ddf2db9f2626cc6.svg
Furong Huang Lab at UMD
user
furongh-lab
67f88bbaf1410163f7c3b692
Lijuan Wang
admin_assigned
2025-04-11T08:34:25.295Z
6413521d4e5305c14f22e110
/avatars/a6f8d0573e678f79bc3c0b7897b818ce.svg
Lijuan Wang
user
Lijuan
SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual
  Reasoning Self-Improvement
In this paper, we present an effective method to enhance visual reasoning
with significantly fewer training samples, relying purely on self-improvement
with no knowledge distillation. Our key insight is that the difficulty of
training data during reinforcement fine-tuning (RFT) is critical. Appropriately
challenging samples can substantially boost reasoning capabilities even when
the dataset is small. Despite being intuitive, the main challenge remains in
accurately quantifying sample difficulty to enable effective data filtering. To
this end, we propose a novel way of repurposing Monte Carlo Tree Search (MCTS)
to achieve that. Starting from our curated 70k open-source training samples, we
introduce an MCTS-based selection method that quantifies sample difficulty
based on the number of iterations required by the VLMs to solve each problem.
This explicit step-by-step reasoning in MCTS enforces the model to think longer
and better identifies samples that are genuinely challenging. We filter and
retain 11k samples to perform RFT on Qwen2.5-VL-7B-Instruct, resulting in our
final model, ThinkLite-VL. Evaluation results on eight benchmarks show that
ThinkLite-VL improves the average performance of Qwen2.5-VL-7B-Instruct by 7%,
using only 11k training samples with no knowledge distillation. This
significantly outperforms all existing 7B-level reasoning VLMs, and our fairly
comparable baselines that use classic selection methods such as accuracy-based
filtering. Notably, on MathVista, ThinkLite-VL-7B achieves the SoTA accuracy of
75.1, surpassing Qwen2.5-VL-72B, GPT-4o, and O1. Our code, data, and model are
available at https://github.com/si0wang/ThinkLite-VL.
A method using Monte Carlo Tree Search for selecting challenging training samples enhances visual reasoning capabilities in a smaller dataset compared to existing models.
reinforcement fine-tuning Monte Carlo Tree Search sample difficulty VLMs visual reasoning Qwen2.5-VL-7B-Instruct ThinkLite-VL MathVista SoTA accuracy