RePIC: Reinforced Post-Training for Personalizing Multi-Modal Language
  Models
Recent multi-modal large language models (MLLMs) often struggle to generate
personalized image captions, even when trained on high-quality captions. In
this work, we observe that such limitations persist in existing
post-training-based MLLM personalization methods. Specifically, despite being
post-tuned with large-scale caption data through supervised fine-tuning (SFT),
these models frequently fail to produce faithful descriptions in real-world
scenarios, such as multi-concept image captioning. However, acquiring
large-scale, high-quality captions for such complex settings is both costly and
difficult. To address the data-centric nature of SFT, we propose a
reinforcement learning (RL)-based post-training framework. To the best of our
knowledge, this is the first RL-based approach to post-train MLLMs for
personalized image captioning. Our method significantly enhances both visual
recognition and personalized generation capabilities of MLLMs, and consistently
outperforms existing SFT-based baselines, especially in the challenging
multi-concept image captioning task.
685b1e9bd2ee4fac76521d87
Yeongtak Oh
claimed_verified
2025-07-18T12:31:53.076Z
636f6e8a31af06da86499ebc
/avatars/9430fbc05774aad8e46c0861769b3c30.svg
Yeongtak
user
Yeongtak
685b1e9bd2ee4fac76521d88
Jisoo Mok
685b1e9bd2ee4fac76521d89
Dohyun Chung
685b1e9bd2ee4fac76521d8a
Juhyeon Shin
685b1e9bd2ee4fac76521d8b
Sangha Park
685b1e9bd2ee4fac76521d8c
Johan Barthelemy
685b1e9bd2ee4fac76521d8d
Sungroh Yoon
RePIC: Reinforced Post-Training for Personalizing Multi-Modal Language
  Models
Recent multi-modal large language models (MLLMs) often struggle to generate
personalized image captions, even when trained on high-quality captions. In
this work, we observe that such limitations persist in existing
post-training-based MLLM personalization methods. Specifically, despite being
post-tuned with large-scale caption data through supervised fine-tuning (SFT),
these models frequently fail to produce faithful descriptions in real-world
scenarios, such as multi-concept image captioning. However, acquiring
large-scale, high-quality captions for such complex settings is both costly and
difficult. To address the data-centric nature of SFT, we propose a
reinforcement learning (RL)-based post-training framework. To the best of our
knowledge, this is the first RL-based approach to post-train MLLMs for
personalized image captioning. Our method significantly enhances both visual
recognition and personalized generation capabilities of MLLMs, and consistently
outperforms existing SFT-based baselines, especially in the challenging
multi-concept image captioning task.
A reinforcement learning-based post-training framework improves the personalized image captioning capabilities of multi-modal large language models compared to supervised fine-tuning methods.
multi-modal large language models personalized image captions supervised fine-tuning reinforcement learning visual recognition