Personalized Multimodal Large Language Models: A Survey
Multimodal Large Language Models (MLLMs) have become increasingly important
due to their state-of-the-art performance and ability to integrate multiple
data modalities, such as text, images, and audio, to perform complex tasks with
high accuracy. This paper presents a comprehensive survey on personalized
multimodal large language models, focusing on their architecture, training
methods, and applications. We propose an intuitive taxonomy for categorizing
the techniques used to personalize MLLMs to individual users, and discuss the
techniques accordingly. Furthermore, we discuss how such techniques can be
combined or adapted when appropriate, highlighting their advantages and
underlying rationale. We also provide a succinct summary of personalization
tasks investigated in existing research, along with the evaluation metrics
commonly used. Additionally, we summarize the datasets that are useful for
benchmarking personalized MLLMs. Finally, we outline critical open challenges.
This survey aims to serve as a valuable resource for researchers and
practitioners seeking to understand and advance the development of personalized
multimodal large language models.
675286e49106cbed4a723624
Junda Wu
675286e49106cbed4a723625
Hanjia Lyu
claimed_verified
2024-12-09T11:07:15.993Z
64c939307dba66c3a7e4d215
/avatars/0b662cc1799525188476f3e6e1f97d29.svg
BruceLyu
user
brucelyu
675286e49106cbed4a723626
Yu Xia
675286e49106cbed4a723627
Zhehao Zhang
675286e49106cbed4a723628
Joe Barrow
675286e49106cbed4a723629
Ishita Kumar
675286e49106cbed4a72362a
Mehrnoosh Mirtaheri
675286e49106cbed4a72362b
Hongjie Chen
675286e49106cbed4a72362c
Ryan A. Rossi
675286e49106cbed4a72362d
Franck Dernoncourt
claimed_verified
2024-12-06T09:08:51.410Z
62c5947524171688a9feb992
/avatars/5a151713b9eae8dc566f5957acee3475.svg
Franck Dernoncourt
user
Franck-Dernoncourt
675286e49106cbed4a72362e
Tong Yu
675286e49106cbed4a72362f
Ruiyi Zhang
675286e49106cbed4a723630
Jiuxiang Gu
675286e49106cbed4a723631
Nesreen K. Ahmed
claimed_verified
2025-01-07T08:42:59.963Z
663e588ccca86c8371a913d9
https://cdn-avatars.huggingface.co/v1/production/uploads/663e588ccca86c8371a913d9/wgVGw5U77cqwtjovhsXyR.jpeg
Nesreen Ahmed
user
nkahmed
675286e49106cbed4a723632
Yu Wang
675286e49106cbed4a723633
Xiang Chen
675286e49106cbed4a723634
Hanieh Deilamsalehy
675286e49106cbed4a723635
Namyong Park
675286e49106cbed4a723636
Sungchul Kim
675286e49106cbed4a723637
Huanrui Yang
675286e49106cbed4a723638
Subrata Mitra
claimed_verified
2025-02-25T09:41:14.441Z
67bd6337daeb37766818ea27
/avatars/63390753d7a3301b48098f9f4ea1a383.svg
Subrata Mitra
user
subrata-mitra
675286e49106cbed4a723639
Zhengmian Hu
admin_assigned
2024-12-06T17:43:07.604Z
651f89f53f889550902a4bd2
/avatars/637759e4db380487c96f4a51c5eda454.svg
Zhengmian Hu
user
xiaoniu-578fa
675286e49106cbed4a72363a
Nedim Lipka
675286e49106cbed4a72363b
Dang Nguyen
675286e49106cbed4a72363c
Yue Zhao
675286e49106cbed4a72363d
Jiebo Luo
675286e49106cbed4a72363e
Julian McAuley
Personalized Multimodal Large Language Models: A Survey
Multimodal Large Language Models (MLLMs) have become increasingly important
due to their state-of-the-art performance and ability to integrate multiple
data modalities, such as text, images, and audio, to perform complex tasks with
high accuracy. This paper presents a comprehensive survey on personalized
multimodal large language models, focusing on their architecture, training
methods, and applications. We propose an intuitive taxonomy for categorizing
the techniques used to personalize MLLMs to individual users, and discuss the
techniques accordingly. Furthermore, we discuss how such techniques can be
combined or adapted when appropriate, highlighting their advantages and
underlying rationale. We also provide a succinct summary of personalization
tasks investigated in existing research, along with the evaluation metrics
commonly used. Additionally, we summarize the datasets that are useful for
benchmarking personalized MLLMs. Finally, we outline critical open challenges.
This survey aims to serve as a valuable resource for researchers and
practitioners seeking to understand and advance the development of personalized
multimodal large language models.
This survey covers the architecture, training methods, and applications of personalized multimodal large language models, proposing a taxonomy for personalization techniques and discussing relevant evaluation metrics and datasets.
Multimodal Large Language Models MLLMs personalized MLLMs architecture training methods evaluation metrics datasets personalization techniques multimodal text images audio open challenges