OmniAlign-V: Towards Enhanced Alignment of MLLMs with Human Preference
Recent advancements in open-source multi-modal large language models (MLLMs)
have primarily focused on enhancing foundational capabilities, leaving a
significant gap in human preference alignment. This paper introduces
OmniAlign-V, a comprehensive dataset of 200K high-quality training samples
featuring diverse images, complex questions, and varied response formats to
improve MLLMs' alignment with human preferences. We also present MM-AlignBench,
a human-annotated benchmark specifically designed to evaluate MLLMs' alignment
with human values. Experimental results show that finetuning MLLMs with
OmniAlign-V, using Supervised Fine-Tuning (SFT) or Direct Preference
Optimization (DPO), significantly enhances human preference alignment while
maintaining or enhancing performance on standard VQA benchmarks, preserving
their fundamental capabilities. Our datasets, benchmark, code and checkpoints
have been released at https://github.com/PhoenixZ810/OmniAlign-V.
67be834ae7b05f9e43b172b2
Xiangyu Zhao
claimed_verified
2025-02-26T08:26:02.247Z
6530e62f536dbca918e71c3e
/avatars/efc93bc767e561c6c6d429f65c23382d.svg
Xiangyu Z
user
PhoenixZ
67be834ae7b05f9e43b172b3
Shengyuan Ding
claimed_verified
2025-02-26T08:25:59.887Z
646cd947da8e99940b6e55cf
https://cdn-avatars.huggingface.co/v1/production/uploads/646cd947da8e99940b6e55cf/9c0P0WppFqNW9pdo8LgOS.jpeg
Shengyuan Ding
user
ChrisDing1105
67be834ae7b05f9e43b172b4
Zicheng Zhang
admin_assigned
2025-02-26T08:49:10.028Z
675aa937ab6aa7ecd09341ce
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/d_CNUsNOw92pg7MVhf9Vm.png
Zicheng Zhang
user
UniverseCA
67be834ae7b05f9e43b172b5
Haian Huang
67be834ae7b05f9e43b172b6
Maosong Cao
67be834ae7b05f9e43b172b7
Weiyun Wang
admin_assigned
2025-02-26T08:48:45.520Z
619507e7b74b6c591f794340
https://cdn-avatars.huggingface.co/v1/production/uploads/619507e7b74b6c591f794340/JbPDoy6Ko1V1-6oJJwFV8.jpeg
Weiyun Wang
user
Weiyun1025
67be834ae7b05f9e43b172b8
Jiaqi Wang
admin_assigned
2025-02-26T08:48:38.876Z
64638c4d51fa6e63060521b5
/avatars/c863ace5b1dc788a341bcf4ddbdfaec1.svg
JIaqi
user
Jiaqiwang
67be834ae7b05f9e43b172b9
Xinyu Fang
claimed_verified
2025-02-26T08:26:04.433Z
64f5f8dd9b17cd59c453c57f
https://cdn-avatars.huggingface.co/v1/production/uploads/64f5f8dd9b17cd59c453c57f/MulhwLcePFUWUQel8LQZ8.jpeg
Xinyu Fang
user
nebulae09
67be834ae7b05f9e43b172ba
Wenhai Wang
admin_assigned
2025-02-26T08:48:28.151Z
64d1c560c0c627dfa71bdbe0
/avatars/f42794fe25bffcd870a1bcee69b95298.svg
wenhai.wang
user
wangwhcore
67be834ae7b05f9e43b172bb
Guangtao Zhai
67be834ae7b05f9e43b172bc
Haodong Duan
admin_assigned
2025-02-26T08:48:20.155Z
63ee1379190ddd6214efd73a
https://cdn-avatars.huggingface.co/v1/production/uploads/1676546883247-noauth.png
HAODONG DUAN
user
KennyUTC
67be834ae7b05f9e43b172bd
Hua Yang
67be834ae7b05f9e43b172be
Kai Chen
OmniAlign-V: Towards Enhanced Alignment of MLLMs with Human Preference
Recent advancements in open-source multi-modal large language models (MLLMs)
have primarily focused on enhancing foundational capabilities, leaving a
significant gap in human preference alignment. This paper introduces
OmniAlign-V, a comprehensive dataset of 200K high-quality training samples
featuring diverse images, complex questions, and varied response formats to
improve MLLMs' alignment with human preferences. We also present MM-AlignBench,
a human-annotated benchmark specifically designed to evaluate MLLMs' alignment
with human values. Experimental results show that finetuning MLLMs with
OmniAlign-V, using Supervised Fine-Tuning (SFT) or Direct Preference
Optimization (DPO), significantly enhances human preference alignment while
maintaining or enhancing performance on standard VQA benchmarks, preserving
their fundamental capabilities. Our datasets, benchmark, code and checkpoints
have been released at https://github.com/PhoenixZ810/OmniAlign-V.
OmniAlign-V improves human preference alignment in MLLMs through supervised fine-tuning and preference optimization without compromising performance on VQA benchmarks.
multi-modal large language models OmniAlign-V MM-AlignBench Supervised Fine-Tuning Direct Preference Optimization human preference alignment VQA benchmarks