URECA: Unique Region Caption Anything
Region-level captioning aims to generate natural language descriptions for
specific image regions while highlighting their distinguishing features.
However, existing methods struggle to produce unique captions across
multi-granularity, limiting their real-world applicability. To address the need
for detailed region-level understanding, we introduce URECA dataset, a
large-scale dataset tailored for multi-granularity region captioning. Unlike
prior datasets that focus primarily on salient objects, URECA dataset ensures a
unique and consistent mapping between regions and captions by incorporating a
diverse set of objects, parts, and background elements. Central to this is a
stage-wise data curation pipeline, where each stage incrementally refines
region selection and caption generation. By leveraging Multimodal Large
Language Models (MLLMs) at each stage, our pipeline produces distinctive and
contextually grounded captions with improved accuracy and semantic diversity.
Building upon this dataset, we present URECA, a novel captioning model designed
to effectively encode multi-granularity regions. URECA maintains essential
spatial properties such as position and shape through simple yet impactful
modifications to existing MLLMs, enabling fine-grained and semantically rich
region descriptions. Our approach introduces dynamic mask modeling and a
high-resolution mask encoder to enhance caption uniqueness. Experiments show
that URECA achieves state-of-the-art performance on URECA dataset and
generalizes well to existing region-level captioning benchmarks.
67f495437e1624ebbaf2d90e
Sangbeom Lim
claimed_verified
2025-04-08T06:51:37.341Z
64842d1edc475c315e41123a
https://cdn-avatars.huggingface.co/v1/production/uploads/64842d1edc475c315e41123a/Yqam_dy0MnQm4Rn1_1_4N.jpeg
Sangbeom Lim
user
SammyLim
67f495437e1624ebbaf2d90f
Junwan Kim
claimed_verified
2025-04-08T06:51:34.302Z
64c2c45ae818eec6128fdda3
/avatars/d4399e25e6399345e263c7902789047e.svg
Jun-Wan KIM
user
junwann
67f495437e1624ebbaf2d910
Heeji Yoon
67f495437e1624ebbaf2d911
Jaewoo Jung
claimed_verified
2025-04-08T06:51:32.321Z
65d02dc017e2b305e0d7bf4f
/avatars/2a50fd0541e7b0e200c577a661956696.svg
Jaewoo Jung
user
crepejung00
67f495437e1624ebbaf2d912
Seungryong Kim
admin_assigned
2025-04-08T09:13:34.541Z
65cf717450818a335a1d3021
/avatars/382a0e0f40f661cda1b2531e3e6ea2ee.svg
Seungryong Kim
user
seungryong
URECA: Unique Region Caption Anything
Region-level captioning aims to generate natural language descriptions for
specific image regions while highlighting their distinguishing features.
However, existing methods struggle to produce unique captions across
multi-granularity, limiting their real-world applicability. To address the need
for detailed region-level understanding, we introduce URECA dataset, a
large-scale dataset tailored for multi-granularity region captioning. Unlike
prior datasets that focus primarily on salient objects, URECA dataset ensures a
unique and consistent mapping between regions and captions by incorporating a
diverse set of objects, parts, and background elements. Central to this is a
stage-wise data curation pipeline, where each stage incrementally refines
region selection and caption generation. By leveraging Multimodal Large
Language Models (MLLMs) at each stage, our pipeline produces distinctive and
contextually grounded captions with improved accuracy and semantic diversity.
Building upon this dataset, we present URECA, a novel captioning model designed
to effectively encode multi-granularity regions. URECA maintains essential
spatial properties such as position and shape through simple yet impactful
modifications to existing MLLMs, enabling fine-grained and semantically rich
region descriptions. Our approach introduces dynamic mask modeling and a
high-resolution mask encoder to enhance caption uniqueness. Experiments show
that URECA achieves state-of-the-art performance on URECA dataset and
generalizes well to existing region-level captioning benchmarks.
The URECA dataset and model address multi-granularity region captioning by ensuring unique and contextually grounded captions through a stage-wise pipeline and dynamic mask modeling.
region-level captioning URECA dataset multimodal large language models MLMLs stage-wise data curation pipeline dynamic mask modeling high-resolution mask encoder spatial properties