title: CLS-RL: Image Classification with Rule-Based Reinforcement Learning
summary: Classification is a core task in machine learning. Recent research has shown
  that although Multimodal Large Language Models (MLLMs) are initially poor at
  image classification, fine-tuning them with an adequate amount of data can
  significantly enhance their performance, making them comparable to SOTA
  classification models. However, acquiring large-scale labeled data is
  expensive. In this paper, we explore few-shot MLLM classification fine-tuning.
  We found that SFT can cause severe overfitting issues and may even degrade
  performance over the zero-shot approach. To address this challenge, inspired by
  the recent successes in rule-based reinforcement learning, we propose CLS-RL,
  which uses verifiable signals as reward to fine-tune MLLMs. We discovered that
  CLS-RL outperforms SFT in most datasets and has a much higher average accuracy
  on both base-to-new and few-shot learning setting. Moreover, we observed a
  free-lunch phenomenon for CLS-RL; when models are fine-tuned on a particular
  dataset, their performance on other distinct datasets may also improve over
  zero-shot models, even if those datasets differ in distribution and class
  names. This suggests that RL-based methods effectively teach models the
  fundamentals of classification. Lastly, inspired by recent works in inference
  time thinking, we re-examine the `thinking process' during fine-tuning, a
  critical aspect of RL-based methods, in the context of visual classification.
  We question whether such tasks require extensive thinking process during
  fine-tuning, proposing that this may actually detract from performance. Based
  on this premise, we introduce the No-Thinking-CLS-RL method, which minimizes
  thinking processes during training by setting an equality accuracy reward. Our
  findings indicate that, with much less fine-tuning time, No-Thinking-CLS-RL
  method achieves superior in-domain performance and generalization capabilities
  than CLS-RL.
paper_authors._id: 67dce5afbabeda89ca6071c3
paper_authors.name: Ming Li
paper_authors._id: 67dce5afbabeda89ca6071c4
paper_authors.name: Shitian Zhao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-21T11:40:22.815Z
paper_authors._id: 62c66504031996c36c86976a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62c66504031996c36c86976a/wIq0YJhkWnEhlzsh-TGYO.png
paper_authors.fullname: steve z
paper_authors.type: user
paper_authors.user: stzhao
paper_authors._id: 67dce5afbabeda89ca6071c5
paper_authors.name: Jike Zhong
paper_authors._id: 67dce5afbabeda89ca6071c6
paper_authors.name: Yuxiang Lai
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-21T11:40:25.107Z
paper_authors._id: 671a569a97cb6656e4859ea8
paper_authors.avatarUrl: /avatars/9eb7966b3ff39ee342e9e681256c86a7.svg
paper_authors.fullname: Yuxiang Lai
paper_authors.type: user
paper_authors.user: yuxianglai117
paper_authors._id: 67dce5afbabeda89ca6071c7
paper_authors.name: Kaipeng Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-02T07:52:10.406Z
paper_authors._id: 65f1713552c38a91e0a445e8
paper_authors.avatarUrl: /avatars/47ab3ada51c9b9976ac1cd0c4301c373.svg
paper_authors.fullname: kaipeng
paper_authors.type: user
paper_authors.user: kpzhang996
paper_title: CLS-RL: Image Classification with Rule-Based Reinforcement Learning
paper_summary: Classification is a core task in machine learning. Recent research has shown
  that although Multimodal Large Language Models (MLLMs) are initially poor at
  image classification, fine-tuning them with an adequate amount of data can
  significantly enhance their performance, making them comparable to SOTA
  classification models. However, acquiring large-scale labeled data is
  expensive. In this paper, we explore few-shot MLLM classification fine-tuning.
  We found that SFT can cause severe overfitting issues and may even degrade
  performance over the zero-shot approach. To address this challenge, inspired by
  the recent successes in rule-based reinforcement learning, we propose CLS-RL,
  which uses verifiable signals as reward to fine-tune MLLMs. We discovered that
  CLS-RL outperforms SFT in most datasets and has a much higher average accuracy
  on both base-to-new and few-shot learning setting. Moreover, we observed a
  free-lunch phenomenon for CLS-RL; when models are fine-tuned on a particular
  dataset, their performance on other distinct datasets may also improve over
  zero-shot models, even if those datasets differ in distribution and class
  names. This suggests that RL-based methods effectively teach models the
  fundamentals of classification. Lastly, inspired by recent works in inference
  time thinking, we re-examine the `thinking process' during fine-tuning, a
  critical aspect of RL-based methods, in the context of visual classification.
  We question whether such tasks require extensive thinking process during
  fine-tuning, proposing that this may actually detract from performance. Based
  on this premise, we introduce the No-Thinking-CLS-RL method, which minimizes
  thinking processes during training by setting an equality accuracy reward. Our
  findings indicate that, with much less fine-tuning time, No-Thinking-CLS-RL
  method achieves superior in-domain performance and generalization capabilities
  than CLS-RL.
paper_ai_summary: CLS-RL, a reinforcement learning-based method using verifiable signals as rewards, outperforms standard fine-tuning for MLLMs in few-shot image classification and demonstrates improved generalization to other datasets, while No-Thinking-CLS-RL further enhances performance with minimal thinking processes during training.
paper_ai_keywords: Multimodal Large Language Models few-shot fine-tuning SFT rule-based reinforcement learning CLS-RL few-shot learning zero-shot models free-lunch phenomenon inference time thinking No-Thinking-CLS-RL