title: Personalized Multimodal Large Language Models: A Survey
summary: Multimodal Large Language Models (MLLMs) have become increasingly important
  due to their state-of-the-art performance and ability to integrate multiple
  data modalities, such as text, images, and audio, to perform complex tasks with
  high accuracy. This paper presents a comprehensive survey on personalized
  multimodal large language models, focusing on their architecture, training
  methods, and applications. We propose an intuitive taxonomy for categorizing
  the techniques used to personalize MLLMs to individual users, and discuss the
  techniques accordingly. Furthermore, we discuss how such techniques can be
  combined or adapted when appropriate, highlighting their advantages and
  underlying rationale. We also provide a succinct summary of personalization
  tasks investigated in existing research, along with the evaluation metrics
  commonly used. Additionally, we summarize the datasets that are useful for
  benchmarking personalized MLLMs. Finally, we outline critical open challenges.
  This survey aims to serve as a valuable resource for researchers and
  practitioners seeking to understand and advance the development of personalized
  multimodal large language models.
paper_authors._id: 675286e49106cbed4a723624
paper_authors.name: Junda Wu
paper_authors._id: 675286e49106cbed4a723625
paper_authors.name: Hanjia Lyu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-09T11:07:15.993Z
paper_authors._id: 64c939307dba66c3a7e4d215
paper_authors.avatarUrl: /avatars/0b662cc1799525188476f3e6e1f97d29.svg
paper_authors.fullname: BruceLyu
paper_authors.type: user
paper_authors.user: brucelyu
paper_authors._id: 675286e49106cbed4a723626
paper_authors.name: Yu Xia
paper_authors._id: 675286e49106cbed4a723627
paper_authors.name: Zhehao Zhang
paper_authors._id: 675286e49106cbed4a723628
paper_authors.name: Joe Barrow
paper_authors._id: 675286e49106cbed4a723629
paper_authors.name: Ishita Kumar
paper_authors._id: 675286e49106cbed4a72362a
paper_authors.name: Mehrnoosh Mirtaheri
paper_authors._id: 675286e49106cbed4a72362b
paper_authors.name: Hongjie Chen
paper_authors._id: 675286e49106cbed4a72362c
paper_authors.name: Ryan A. Rossi
paper_authors._id: 675286e49106cbed4a72362d
paper_authors.name: Franck Dernoncourt
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-06T09:08:51.410Z
paper_authors._id: 62c5947524171688a9feb992
paper_authors.avatarUrl: /avatars/5a151713b9eae8dc566f5957acee3475.svg
paper_authors.fullname: Franck Dernoncourt
paper_authors.type: user
paper_authors.user: Franck-Dernoncourt
paper_authors._id: 675286e49106cbed4a72362e
paper_authors.name: Tong Yu
paper_authors._id: 675286e49106cbed4a72362f
paper_authors.name: Ruiyi Zhang
paper_authors._id: 675286e49106cbed4a723630
paper_authors.name: Jiuxiang Gu
paper_authors._id: 675286e49106cbed4a723631
paper_authors.name: Nesreen K. Ahmed
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-07T08:42:59.963Z
paper_authors._id: 663e588ccca86c8371a913d9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/663e588ccca86c8371a913d9/wgVGw5U77cqwtjovhsXyR.jpeg
paper_authors.fullname: Nesreen Ahmed
paper_authors.type: user
paper_authors.user: nkahmed
paper_authors._id: 675286e49106cbed4a723632
paper_authors.name: Yu Wang
paper_authors._id: 675286e49106cbed4a723633
paper_authors.name: Xiang Chen
paper_authors._id: 675286e49106cbed4a723634
paper_authors.name: Hanieh Deilamsalehy
paper_authors._id: 675286e49106cbed4a723635
paper_authors.name: Namyong Park
paper_authors._id: 675286e49106cbed4a723636
paper_authors.name: Sungchul Kim
paper_authors._id: 675286e49106cbed4a723637
paper_authors.name: Huanrui Yang
paper_authors._id: 675286e49106cbed4a723638
paper_authors.name: Subrata Mitra
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-25T09:41:14.441Z
paper_authors._id: 67bd6337daeb37766818ea27
paper_authors.avatarUrl: /avatars/63390753d7a3301b48098f9f4ea1a383.svg
paper_authors.fullname: Subrata Mitra
paper_authors.type: user
paper_authors.user: subrata-mitra
paper_authors._id: 675286e49106cbed4a723639
paper_authors.name: Zhengmian Hu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-06T17:43:07.604Z
paper_authors._id: 651f89f53f889550902a4bd2
paper_authors.avatarUrl: /avatars/637759e4db380487c96f4a51c5eda454.svg
paper_authors.fullname: Zhengmian Hu
paper_authors.type: user
paper_authors.user: xiaoniu-578fa
paper_authors._id: 675286e49106cbed4a72363a
paper_authors.name: Nedim Lipka
paper_authors._id: 675286e49106cbed4a72363b
paper_authors.name: Dang Nguyen
paper_authors._id: 675286e49106cbed4a72363c
paper_authors.name: Yue Zhao
paper_authors._id: 675286e49106cbed4a72363d
paper_authors.name: Jiebo Luo
paper_authors._id: 675286e49106cbed4a72363e
paper_authors.name: Julian McAuley
paper_title: Personalized Multimodal Large Language Models: A Survey
paper_summary: Multimodal Large Language Models (MLLMs) have become increasingly important
  due to their state-of-the-art performance and ability to integrate multiple
  data modalities, such as text, images, and audio, to perform complex tasks with
  high accuracy. This paper presents a comprehensive survey on personalized
  multimodal large language models, focusing on their architecture, training
  methods, and applications. We propose an intuitive taxonomy for categorizing
  the techniques used to personalize MLLMs to individual users, and discuss the
  techniques accordingly. Furthermore, we discuss how such techniques can be
  combined or adapted when appropriate, highlighting their advantages and
  underlying rationale. We also provide a succinct summary of personalization
  tasks investigated in existing research, along with the evaluation metrics
  commonly used. Additionally, we summarize the datasets that are useful for
  benchmarking personalized MLLMs. Finally, we outline critical open challenges.
  This survey aims to serve as a valuable resource for researchers and
  practitioners seeking to understand and advance the development of personalized
  multimodal large language models.
paper_ai_keywords: multimodal large language models (MLLMs) personalization techniques taxonomy training methods applications evaluation metrics datasets benchmarking open challenges