title: OmniAlign-V: Towards Enhanced Alignment of MLLMs with Human Preference
summary: Recent advancements in open-source multi-modal large language models (MLLMs)
  have primarily focused on enhancing foundational capabilities, leaving a
  significant gap in human preference alignment. This paper introduces
  OmniAlign-V, a comprehensive dataset of 200K high-quality training samples
  featuring diverse images, complex questions, and varied response formats to
  improve MLLMs' alignment with human preferences. We also present MM-AlignBench,
  a human-annotated benchmark specifically designed to evaluate MLLMs' alignment
  with human values. Experimental results show that finetuning MLLMs with
  OmniAlign-V, using Supervised Fine-Tuning (SFT) or Direct Preference
  Optimization (DPO), significantly enhances human preference alignment while
  maintaining or enhancing performance on standard VQA benchmarks, preserving
  their fundamental capabilities. Our datasets, benchmark, code and checkpoints
  have been released at https://github.com/PhoenixZ810/OmniAlign-V.
paper_authors._id: 67be834ae7b05f9e43b172b2
paper_authors.name: Xiangyu Zhao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-26T08:26:02.247Z
paper_authors._id: 6530e62f536dbca918e71c3e
paper_authors.avatarUrl: /avatars/efc93bc767e561c6c6d429f65c23382d.svg
paper_authors.fullname: Xiangyu Z
paper_authors.type: user
paper_authors.user: PhoenixZ
paper_authors._id: 67be834ae7b05f9e43b172b3
paper_authors.name: Shengyuan Ding
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-26T08:25:59.887Z
paper_authors._id: 646cd947da8e99940b6e55cf
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/646cd947da8e99940b6e55cf/9c0P0WppFqNW9pdo8LgOS.jpeg
paper_authors.fullname: Shengyuan Ding
paper_authors.type: user
paper_authors.user: ChrisDing1105
paper_authors._id: 67be834ae7b05f9e43b172b4
paper_authors.name: Zicheng Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-26T08:49:10.028Z
paper_authors._id: 675aa937ab6aa7ecd09341ce
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/d_CNUsNOw92pg7MVhf9Vm.png
paper_authors.fullname: Zicheng Zhang
paper_authors.type: user
paper_authors.user: UniverseCA
paper_authors._id: 67be834ae7b05f9e43b172b5
paper_authors.name: Haian Huang
paper_authors._id: 67be834ae7b05f9e43b172b6
paper_authors.name: Maosong Cao
paper_authors._id: 67be834ae7b05f9e43b172b7
paper_authors.name: Weiyun Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-26T08:48:45.520Z
paper_authors._id: 619507e7b74b6c591f794340
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/619507e7b74b6c591f794340/JbPDoy6Ko1V1-6oJJwFV8.jpeg
paper_authors.fullname: Weiyun Wang
paper_authors.type: user
paper_authors.user: Weiyun1025
paper_authors._id: 67be834ae7b05f9e43b172b8
paper_authors.name: Jiaqi Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-26T08:48:38.876Z
paper_authors._id: 64638c4d51fa6e63060521b5
paper_authors.avatarUrl: /avatars/c863ace5b1dc788a341bcf4ddbdfaec1.svg
paper_authors.fullname: JIaqi
paper_authors.type: user
paper_authors.user: Jiaqiwang
paper_authors._id: 67be834ae7b05f9e43b172b9
paper_authors.name: Xinyu Fang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-26T08:26:04.433Z
paper_authors._id: 64f5f8dd9b17cd59c453c57f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64f5f8dd9b17cd59c453c57f/MulhwLcePFUWUQel8LQZ8.jpeg
paper_authors.fullname: Xinyu Fang
paper_authors.type: user
paper_authors.user: nebulae09
paper_authors._id: 67be834ae7b05f9e43b172ba
paper_authors.name: Wenhai Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-26T08:48:28.151Z
paper_authors._id: 64d1c560c0c627dfa71bdbe0
paper_authors.avatarUrl: /avatars/f42794fe25bffcd870a1bcee69b95298.svg
paper_authors.fullname: wenhai.wang
paper_authors.type: user
paper_authors.user: wangwhcore
paper_authors._id: 67be834ae7b05f9e43b172bb
paper_authors.name: Guangtao Zhai
paper_authors._id: 67be834ae7b05f9e43b172bc
paper_authors.name: Haodong Duan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-26T08:48:20.155Z
paper_authors._id: 63ee1379190ddd6214efd73a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1676546883247-noauth.png
paper_authors.fullname: HAODONG DUAN
paper_authors.type: user
paper_authors.user: KennyUTC
paper_authors._id: 67be834ae7b05f9e43b172bd
paper_authors.name: Hua Yang
paper_authors._id: 67be834ae7b05f9e43b172be
paper_authors.name: Kai Chen
paper_title: OmniAlign-V: Towards Enhanced Alignment of MLLMs with Human Preference
paper_summary: Recent advancements in open-source multi-modal large language models (MLLMs)
  have primarily focused on enhancing foundational capabilities, leaving a
  significant gap in human preference alignment. This paper introduces
  OmniAlign-V, a comprehensive dataset of 200K high-quality training samples
  featuring diverse images, complex questions, and varied response formats to
  improve MLLMs' alignment with human preferences. We also present MM-AlignBench,
  a human-annotated benchmark specifically designed to evaluate MLLMs' alignment
  with human values. Experimental results show that finetuning MLLMs with
  OmniAlign-V, using Supervised Fine-Tuning (SFT) or Direct Preference
  Optimization (DPO), significantly enhances human preference alignment while
  maintaining or enhancing performance on standard VQA benchmarks, preserving
  their fundamental capabilities. Our datasets, benchmark, code and checkpoints
  have been released at https://github.com/PhoenixZ810/OmniAlign-V.
paper_ai_keywords: multi-modal large language models (MLLMs) foundational capabilities human preference alignment comprehensive dataset high-quality training samples diverse images complex questions varied response formats MM-AlignBench human-annotated benchmark human values fine-tuning Supervised Fine-Tuning (SFT) Direct Preference Optimization (DPO) standard VQA benchmarks