Aligning Multimodal LLM with Human Preference: A Survey
Large language models (LLMs) can handle a wide variety of general tasks with
simple prompts, without the need for task-specific training. Multimodal Large
Language Models (MLLMs), built upon LLMs, have demonstrated impressive
potential in tackling complex tasks involving visual, auditory, and textual
data. However, critical issues related to truthfulness, safety, o1-like
reasoning, and alignment with human preference remain insufficiently addressed.
This gap has spurred the emergence of various alignment algorithms, each
targeting different application scenarios and optimization goals. Recent
studies have shown that alignment algorithms are a powerful approach to
resolving the aforementioned challenges. In this paper, we aim to provide a
comprehensive and systematic review of alignment algorithms for MLLMs.
Specifically, we explore four key aspects: (1) the application scenarios
covered by alignment algorithms, including general image understanding,
multi-image, video, and audio, and extended multimodal applications; (2) the
core factors in constructing alignment datasets, including data sources, model
responses, and preference annotations; (3) the benchmarks used to evaluate
alignment algorithms; and (4) a discussion of potential future directions for
the development of alignment algorithms. This work seeks to help researchers
organize current advancements in the field and inspire better alignment
methods. The project page of this paper is available at
https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models/tree/Alignment.
67da436711b6db6920802e9e
Tao Yu
67da436711b6db6920802e9f
Yi-Fan Zhang
extracted_confirmed
2025-03-20T03:10:01.854Z
623d8ca4c29adf5ef6175615
https://cdn-avatars.huggingface.co/v1/production/uploads/623d8ca4c29adf5ef6175615/q7lHao7UPwU1u7YLSP56m.jpeg
Yi-Fan Zhang
user
yifanzhang114
67da436711b6db6920802ea0
Chaoyou Fu
67da436711b6db6920802ea1
Junkang Wu
67da436711b6db6920802ea2
Jinda Lu
67da436711b6db6920802ea3
Kun Wang
67da436711b6db6920802ea4
Xingyu Lu
67da436711b6db6920802ea5
Yunhang Shen
admin_assigned
2025-03-19T13:38:55.153Z
6483143902f98c3f05aff915
https://cdn-avatars.huggingface.co/v1/production/uploads/6483143902f98c3f05aff915/ZhWFFgrlRsQf4MXiInh5p.jpeg
沈云航 Yunhang Shen
user
shenyunhang
67da436711b6db6920802ea6
Guibin Zhang
67da436711b6db6920802ea7
Dingjie Song
admin_assigned
2025-03-19T13:38:06.473Z
619f01b8cc04eadf54fa5d5d
/avatars/928f3d1a6146e2e1ae4860445d929d5c.svg
Song Dingjie
user
songdj
67da436711b6db6920802ea8
Yibo Yan
67da436711b6db6920802ea9
Tianlong Xu
admin_assigned
2025-03-19T13:37:52.755Z
66b65be6308cd3b74061c71a
/avatars/e8e9120a25044fde0a9483b4553ab8fa.svg
Tianlong Xu
user
xtl090994
67da436711b6db6920802eaa
Qingsong Wen
67da436711b6db6920802eab
Zhang Zhang
67da436711b6db6920802eac
Yan Huang
67da436711b6db6920802ead
Liang Wang
67da436711b6db6920802eae
Tieniu Tan
Aligning Multimodal LLM with Human Preference: A Survey
Large language models (LLMs) can handle a wide variety of general tasks with
simple prompts, without the need for task-specific training. Multimodal Large
Language Models (MLLMs), built upon LLMs, have demonstrated impressive
potential in tackling complex tasks involving visual, auditory, and textual
data. However, critical issues related to truthfulness, safety, o1-like
reasoning, and alignment with human preference remain insufficiently addressed.
This gap has spurred the emergence of various alignment algorithms, each
targeting different application scenarios and optimization goals. Recent
studies have shown that alignment algorithms are a powerful approach to
resolving the aforementioned challenges. In this paper, we aim to provide a
comprehensive and systematic review of alignment algorithms for MLLMs.
Specifically, we explore four key aspects: (1) the application scenarios
covered by alignment algorithms, including general image understanding,
multi-image, video, and audio, and extended multimodal applications; (2) the
core factors in constructing alignment datasets, including data sources, model
responses, and preference annotations; (3) the benchmarks used to evaluate
alignment algorithms; and (4) a discussion of potential future directions for
the development of alignment algorithms. This work seeks to help researchers
organize current advancements in the field and inspire better alignment
methods. The project page of this paper is available at
https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models/tree/Alignment.
A comprehensive review of alignment algorithms for multimodal large language models, covering their application scenarios, dataset construction, evaluation benchmarks, and future directions.
Large language models (LLMs) Multimodal Large Language Models (MLLMs) alignment algorithms truthfulness safety reasoning human preference image understanding multi-image video audio data sources model responses preference annotations benchmarks future directions