GenRecal: Generation after Recalibration from Large to Small
  Vision-Language Models
Recent advancements in vision-language models (VLMs) have leveraged large
language models (LLMs) to achieve performance on par with closed-source systems
like GPT-4V. However, deploying these models in real-world scenarios,
particularly on resource-constrained devices, remains challenging due to their
substantial computational demands. This has spurred interest in distilling
knowledge from large VLMs into smaller, more efficient counterparts. A key
challenge arises here from the diversity of VLM architectures, which are built
on different LLMs and employ varying token types-differing in vocabulary size,
token splits, and token index ordering. To address this challenge of limitation
to a specific VLM type, we present Generation after Recalibration (GenRecal), a
novel, general-purpose distillation framework for VLMs. GenRecal incorporates a
Recalibrator that aligns and adapts feature representations between
heterogeneous VLMs, enabling effective knowledge transfer across different
types of VLMs. Through extensive experiments on multiple challenging
benchmarks, we demonstrate that GenRecal significantly improves baseline
performances, eventually outperforming large-scale open- and closed-source
VLMs.
68536fc899bf39f9665c7961
Byung-Kwan Lee
claimed_verified
2025-06-19T09:10:25.415Z
657152eb12f162153b50ec9d
https://cdn-avatars.huggingface.co/v1/production/uploads/657152eb12f162153b50ec9d/qnldHP35PclV0pDz_05q8.jpeg
Byung-Kwan Lee
user
BK-Lee
68536fc899bf39f9665c7962
Ryo Hachiuma
extracted_pending
2025-06-19T02:02:49.546Z
65b33e5f7cd0069ad648c4e8
/avatars/1a746ea535cffa92ea08006e05ea414a.svg
Ryo Hachiuma
user
rhachiuma
68536fc899bf39f9665c7963
Yong Man Ro
admin_assigned
2025-06-20T12:28:59.645Z
660603529e3555d648e3baf1
/avatars/0f54479afcfc19df00b25d5aedb4cf67.svg
Yong Man Ro
user
dwightro
68536fc899bf39f9665c7964
Yu-Chiang Frank Wang
68536fc899bf39f9665c7965
Yueh-Hua Wu
GenRecal: Generation after Recalibration from Large to Small
  Vision-Language Models
Recent advancements in vision-language models (VLMs) have leveraged large
language models (LLMs) to achieve performance on par with closed-source systems
like GPT-4V. However, deploying these models in real-world scenarios,
particularly on resource-constrained devices, remains challenging due to their
substantial computational demands. This has spurred interest in distilling
knowledge from large VLMs into smaller, more efficient counterparts. A key
challenge arises here from the diversity of VLM architectures, which are built
on different LLMs and employ varying token types-differing in vocabulary size,
token splits, and token index ordering. To address this challenge of limitation
to a specific VLM type, we present Generation after Recalibration (GenRecal), a
novel, general-purpose distillation framework for VLMs. GenRecal incorporates a
Recalibrator that aligns and adapts feature representations between
heterogeneous VLMs, enabling effective knowledge transfer across different
types of VLMs. Through extensive experiments on multiple challenging
benchmarks, we demonstrate that GenRecal significantly improves baseline
performances, eventually outperforming large-scale open- and closed-source
VLMs.
GenRecal, a novel distillation framework, improves performance of vision-language models by aligning feature representations across different architectures.
vision-language models large language models distillation framework GenerRecal recalibration feature representations heterogeneous VLMs