Molar: Multimodal LLMs with Collaborative Filtering Alignment for
  Enhanced Sequential Recommendation
Sequential recommendation (SR) systems have evolved significantly over the
past decade, transitioning from traditional collaborative filtering to deep
learning approaches and, more recently, to large language models (LLMs). While
the adoption of LLMs has driven substantial advancements, these models
inherently lack collaborative filtering information, relying primarily on
textual content data neglecting other modalities and thus failing to achieve
optimal recommendation performance. To address this limitation, we propose
Molar, a Multimodal large language sequential recommendation framework that
integrates multiple content modalities with ID information to capture
collaborative signals effectively. Molar employs an MLLM to generate unified
item representations from both textual and non-textual data, facilitating
comprehensive multimodal modeling and enriching item embeddings. Additionally,
it incorporates collaborative filtering signals through a post-alignment
mechanism, which aligns user representations from content-based and ID-based
models, ensuring precise personalization and robust performance. By seamlessly
combining multimodal content with collaborative filtering insights, Molar
captures both user interests and contextual semantics, leading to superior
recommendation accuracy. Extensive experiments validate that Molar
significantly outperforms traditional and LLM-based baselines, highlighting its
strength in utilizing multimodal data and collaborative signals for sequential
recommendation tasks. The source code is available at
https://anonymous.4open.science/r/Molar-8B06/.
676e9d9d8126645611b73ecb
Yucong Luo
admin_assigned
2024-12-30T21:11:39.638Z
644679e3eda3580e81a0832b
/avatars/bc7c67955644efe55244f9b8a68b8408.svg
Luo Yucong
user
GodFire
676e9d9d8126645611b73ecc
Qitao Qin
676e9d9d8126645611b73ecd
Hao Zhang
admin_assigned
2024-12-30T21:11:55.544Z
6320ddd4a023aad6a768db54
/avatars/96b449a5cf6b2c7a818d7e7dc8c2e821.svg
Hao Zhang
user
haozhang
676e9d9d8126645611b73ece
Mingyue Cheng
admin_assigned
2024-12-30T21:12:00.921Z
647f5222e9c81260ff87640d
https://cdn-avatars.huggingface.co/v1/production/uploads/647f5222e9c81260ff87640d/u7kmIrFEzaIG2Id6R2bAz.jpeg
Mingyue Cheng
user
twigcheng
676e9d9d8126645611b73ecf
Ruiran Yan
admin_assigned
2024-12-30T21:12:07.779Z
661ac5b53d7248a6f20080c1
/avatars/26aef5944759c2e4366a71eb8c7fc50a.svg
Ruiran Yan
user
Ruiran
676e9d9d8126645611b73ed0
Kefan Wang
676e9d9d8126645611b73ed1
Jie Ouyang
admin_assigned
2024-12-30T21:12:23.124Z
653780f5434f0b412aba51c2
/avatars/1c416e5b178fcd04898a393cc2ef8d6c.svg
Jie Ouyang
user
russwest404
Molar: Multimodal LLMs with Collaborative Filtering Alignment for
  Enhanced Sequential Recommendation
Sequential recommendation (SR) systems have evolved significantly over the
past decade, transitioning from traditional collaborative filtering to deep
learning approaches and, more recently, to large language models (LLMs). While
the adoption of LLMs has driven substantial advancements, these models
inherently lack collaborative filtering information, relying primarily on
textual content data neglecting other modalities and thus failing to achieve
optimal recommendation performance. To address this limitation, we propose
Molar, a Multimodal large language sequential recommendation framework that
integrates multiple content modalities with ID information to capture
collaborative signals effectively. Molar employs an MLLM to generate unified
item representations from both textual and non-textual data, facilitating
comprehensive multimodal modeling and enriching item embeddings. Additionally,
it incorporates collaborative filtering signals through a post-alignment
mechanism, which aligns user representations from content-based and ID-based
models, ensuring precise personalization and robust performance. By seamlessly
combining multimodal content with collaborative filtering insights, Molar
captures both user interests and contextual semantics, leading to superior
recommendation accuracy. Extensive experiments validate that Molar
significantly outperforms traditional and LLM-based baselines, highlighting its
strength in utilizing multimodal data and collaborative signals for sequential
recommendation tasks. The source code is available at
https://anonymous.4open.science/r/Molar-8B06/.
Molar, a multimodal large language sequential recommendation framework, enhances recommendation performance by integrating textual and non-textual data with ID information to capture both collaborative filters and user interests.
Multimodal large language sequential recommendation framework Molar MLLM unified item representations multimodal modeling item embeddings post-alignment mechanism user representations content-based models ID-based models sequential recommendation tasks