QFFT, Question-Free Fine-Tuning for Adaptive Reasoning
Recent advancements in Long Chain-of-Thought (CoT) reasoning models have
improved performance on complex tasks, but they suffer from overthinking, which
generates redundant reasoning steps, especially for simple questions. This
paper revisits the reasoning patterns of Long and Short CoT models, observing
that the Short CoT patterns offer concise reasoning efficiently, while the Long
CoT patterns excel in challenging scenarios where the Short CoT patterns
struggle. To enable models to leverage both patterns, we propose Question-Free
Fine-Tuning (QFFT), a fine-tuning approach that removes the input question
during training and learns exclusively from Long CoT responses. This approach
enables the model to adaptively employ both reasoning patterns: it prioritizes
the Short CoT patterns and activates the Long CoT patterns only when necessary.
Experiments on various mathematical datasets demonstrate that QFFT reduces
average response length by more than 50\%, while achieving performance
comparable to Supervised Fine-Tuning (SFT). Additionally, QFFT exhibits
superior performance compared to SFT in noisy, out-of-domain, and low-resource
scenarios.
685226a40164cd13167104bd
Wanlong Liu
claimed_verified
2025-06-18T10:59:15.053Z
64eb333e6878d90b031fa5c5
/avatars/a0d875b49d1c56be88f34854647306da.svg
Wanlong Liu
user
lwl-uestc
685226a40164cd13167104be
Junxiao Xu
685226a40164cd13167104bf
Fei Yu
685226a40164cd13167104c0
Yukang Lin
685226a40164cd13167104c1
Ke Ji
685226a40164cd13167104c2
Wenyu Chen
685226a40164cd13167104c3
Yan Xu
685226a40164cd13167104c4
Yasheng Wang
685226a40164cd13167104c5
Lifeng Shang
685226a40164cd13167104c6
Benyou Wang
QFFT, Question-Free Fine-Tuning for Adaptive Reasoning
Recent advancements in Long Chain-of-Thought (CoT) reasoning models have
improved performance on complex tasks, but they suffer from overthinking, which
generates redundant reasoning steps, especially for simple questions. This
paper revisits the reasoning patterns of Long and Short CoT models, observing
that the Short CoT patterns offer concise reasoning efficiently, while the Long
CoT patterns excel in challenging scenarios where the Short CoT patterns
struggle. To enable models to leverage both patterns, we propose Question-Free
Fine-Tuning (QFFT), a fine-tuning approach that removes the input question
during training and learns exclusively from Long CoT responses. This approach
enables the model to adaptively employ both reasoning patterns: it prioritizes
the Short CoT patterns and activates the Long CoT patterns only when necessary.
Experiments on various mathematical datasets demonstrate that QFFT reduces
average response length by more than 50\%, while achieving performance
comparable to Supervised Fine-Tuning (SFT). Additionally, QFFT exhibits
superior performance compared to SFT in noisy, out-of-domain, and low-resource
scenarios.
Question-Free Fine-Tuning (QFFT) improves efficiency and adaptability in cognitive models by leveraging both short and long chain-of-thought patterns, reducing response length while maintaining performance across various scenarios.
Long Chain-of-Thought Short Chain-of-Thought Question-Free Fine-Tuning fine-tuning Supervised Fine-Tuning