title: Molar: Multimodal LLMs with Collaborative Filtering Alignment for
    Enhanced Sequential Recommendation
summary: Sequential recommendation (SR) systems have evolved significantly over the
  past decade, transitioning from traditional collaborative filtering to deep
  learning approaches and, more recently, to large language models (LLMs). While
  the adoption of LLMs has driven substantial advancements, these models
  inherently lack collaborative filtering information, relying primarily on
  textual content data neglecting other modalities and thus failing to achieve
  optimal recommendation performance. To address this limitation, we propose
  Molar, a Multimodal large language sequential recommendation framework that
  integrates multiple content modalities with ID information to capture
  collaborative signals effectively. Molar employs an MLLM to generate unified
  item representations from both textual and non-textual data, facilitating
  comprehensive multimodal modeling and enriching item embeddings. Additionally,
  it incorporates collaborative filtering signals through a post-alignment
  mechanism, which aligns user representations from content-based and ID-based
  models, ensuring precise personalization and robust performance. By seamlessly
  combining multimodal content with collaborative filtering insights, Molar
  captures both user interests and contextual semantics, leading to superior
  recommendation accuracy. Extensive experiments validate that Molar
  significantly outperforms traditional and LLM-based baselines, highlighting its
  strength in utilizing multimodal data and collaborative signals for sequential
  recommendation tasks. The source code is available at
  https://anonymous.4open.science/r/Molar-8B06/.
paper_authors._id: 676e9d9d8126645611b73ecb
paper_authors.name: Yucong Luo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-30T21:11:39.638Z
paper_authors._id: 644679e3eda3580e81a0832b
paper_authors.avatarUrl: /avatars/bc7c67955644efe55244f9b8a68b8408.svg
paper_authors.fullname: Luo Yucong
paper_authors.type: user
paper_authors.user: GodFire
paper_authors._id: 676e9d9d8126645611b73ecc
paper_authors.name: Qitao Qin
paper_authors._id: 676e9d9d8126645611b73ecd
paper_authors.name: Hao Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-30T21:11:55.544Z
paper_authors._id: 6320ddd4a023aad6a768db54
paper_authors.avatarUrl: /avatars/96b449a5cf6b2c7a818d7e7dc8c2e821.svg
paper_authors.fullname: Hao Zhang
paper_authors.type: user
paper_authors.user: haozhang
paper_authors._id: 676e9d9d8126645611b73ece
paper_authors.name: Mingyue Cheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-30T21:12:00.921Z
paper_authors._id: 647f5222e9c81260ff87640d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/647f5222e9c81260ff87640d/u7kmIrFEzaIG2Id6R2bAz.jpeg
paper_authors.fullname: Mingyue Cheng
paper_authors.type: user
paper_authors.user: twigcheng
paper_authors._id: 676e9d9d8126645611b73ecf
paper_authors.name: Ruiran Yan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-30T21:12:07.779Z
paper_authors._id: 661ac5b53d7248a6f20080c1
paper_authors.avatarUrl: /avatars/26aef5944759c2e4366a71eb8c7fc50a.svg
paper_authors.fullname: Ruiran Yan
paper_authors.type: user
paper_authors.user: Ruiran
paper_authors._id: 676e9d9d8126645611b73ed0
paper_authors.name: Kefan Wang
paper_authors._id: 676e9d9d8126645611b73ed1
paper_authors.name: Jie Ouyang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-30T21:12:23.124Z
paper_authors._id: 653780f5434f0b412aba51c2
paper_authors.avatarUrl: /avatars/1c416e5b178fcd04898a393cc2ef8d6c.svg
paper_authors.fullname: Jie Ouyang
paper_authors.type: user
paper_authors.user: russwest404
paper_title: Molar: Multimodal LLMs with Collaborative Filtering Alignment for
    Enhanced Sequential Recommendation
paper_summary: Sequential recommendation (SR) systems have evolved significantly over the
  past decade, transitioning from traditional collaborative filtering to deep
  learning approaches and, more recently, to large language models (LLMs). While
  the adoption of LLMs has driven substantial advancements, these models
  inherently lack collaborative filtering information, relying primarily on
  textual content data neglecting other modalities and thus failing to achieve
  optimal recommendation performance. To address this limitation, we propose
  Molar, a Multimodal large language sequential recommendation framework that
  integrates multiple content modalities with ID information to capture
  collaborative signals effectively. Molar employs an MLLM to generate unified
  item representations from both textual and non-textual data, facilitating
  comprehensive multimodal modeling and enriching item embeddings. Additionally,
  it incorporates collaborative filtering signals through a post-alignment
  mechanism, which aligns user representations from content-based and ID-based
  models, ensuring precise personalization and robust performance. By seamlessly
  combining multimodal content with collaborative filtering insights, Molar
  captures both user interests and contextual semantics, leading to superior
  recommendation accuracy. Extensive experiments validate that Molar
  significantly outperforms traditional and LLM-based baselines, highlighting its
  strength in utilizing multimodal data and collaborative signals for sequential
  recommendation tasks. The source code is available at
  https://anonymous.4open.science/r/Molar-8B06/.
paper_ai_keywords: Sequential recommendation (SR) Large language models (LLMs) Multimodal large language sequential recommendation framework (Molar) Multimodal Unified item representations Non-textual data Item embeddings Collaborative filtering signals Post-alignment mechanism User representations Content-based ID-based models Precise personalization Robust performance Recommendation accuracy Baselines