Region-based Cluster Discrimination for Visual Representation Learning
Learning visual representations is foundational for a broad spectrum of
downstream tasks. Although recent vision-language contrastive models, such as
CLIP and SigLIP, have achieved impressive zero-shot performance via large-scale
vision-language alignment, their reliance on global representations constrains
their effectiveness for dense prediction tasks, such as grounding, OCR, and
segmentation. To address this gap, we introduce Region-Aware Cluster
Discrimination (RICE), a novel method that enhances region-level visual and OCR
capabilities. We first construct a billion-scale candidate region dataset and
propose a Region Transformer layer to extract rich regional semantics. We
further design a unified region cluster discrimination loss that jointly
supports object and OCR learning within a single classification framework,
enabling efficient and scalable distributed training on large-scale data.
Extensive experiments show that RICE consistently outperforms previous methods
on tasks, including segmentation, dense detection, and visual perception for
Multimodal Large Language Models (MLLMs). The pre-trained models have been
released at https://github.com/deepglint/MVT.
68883088af872d625c10c5eb
Yin Xie
68883088af872d625c10c5ec
Kaicheng Yang
claimed_verified
2025-07-29T07:08:26.917Z
63e202f352b7578dba448ab5
https://cdn-avatars.huggingface.co/v1/production/uploads/63e202f352b7578dba448ab5/8itVBLcv14m7OVsoF8h1o.jpeg
Yang
user
Kaichengalex
68883088af872d625c10c5ed
Xiang An
claimed_verified
2025-07-29T07:08:29.232Z
6478679d7b370854241b2ad8
https://cdn-avatars.huggingface.co/v1/production/uploads/6478679d7b370854241b2ad8/dBczWYYdfEt9tQcnVGhQk.jpeg
xiangan
user
xiangan
68883088af872d625c10c5ee
Kun Wu
claimed_verified
2025-07-29T07:08:24.529Z
64f597588b6d053c709debd9
/avatars/85d046a412c503f883e16324030a457b.svg
Kun
user
Athinklo
68883088af872d625c10c5ef
Yongle Zhao
68883088af872d625c10c5f0
Weimo Deng
68883088af872d625c10c5f1
Zimin Ran
68883088af872d625c10c5f2
Yumeng Wang
68883088af872d625c10c5f3
Ziyong Feng
68883088af872d625c10c5f4
Roy Miles
68883088af872d625c10c5f5
Ismail Elezi
68883088af872d625c10c5f6
Jiankang Deng
Region-based Cluster Discrimination for Visual Representation Learning
Learning visual representations is foundational for a broad spectrum of
downstream tasks. Although recent vision-language contrastive models, such as
CLIP and SigLIP, have achieved impressive zero-shot performance via large-scale
vision-language alignment, their reliance on global representations constrains
their effectiveness for dense prediction tasks, such as grounding, OCR, and
segmentation. To address this gap, we introduce Region-Aware Cluster
Discrimination (RICE), a novel method that enhances region-level visual and OCR
capabilities. We first construct a billion-scale candidate region dataset and
propose a Region Transformer layer to extract rich regional semantics. We
further design a unified region cluster discrimination loss that jointly
supports object and OCR learning within a single classification framework,
enabling efficient and scalable distributed training on large-scale data.
Extensive experiments show that RICE consistently outperforms previous methods
on tasks, including segmentation, dense detection, and visual perception for
Multimodal Large Language Models (MLLMs). The pre-trained models have been
released at https://github.com/deepglint/MVT.
RICE, a novel method using a Region Transformer and region cluster discrimination loss, enhances region-level visual and OCR capabilities, outperforming previous methods in tasks like segmentation and dense detection.
Region-Aware Cluster Discrimination RICE Region Transformer region cluster discrimination loss dense prediction tasks grounding OCR segmentation dense detection Multimodal Large Language Models MLLMs