title: Qilin: A Multimodal Information Retrieval Dataset with APP-level User
    Sessions
summary: User-generated content (UGC) communities, especially those featuring
  multimodal content, improve user experiences by integrating visual and textual
  information into results (or items). The challenge of improving user
  experiences in complex systems with search and recommendation (S\&R) services
  has drawn significant attention from both academia and industry these years.
  However, the lack of high-quality datasets has limited the research progress on
  multimodal S\&R. To address the growing need for developing better S\&R
  services, we present a novel multimodal information retrieval dataset in this
  paper, namely Qilin. The dataset is collected from Xiaohongshu, a popular
  social platform with over 300 million monthly active users and an average
  search penetration rate of over 70\%. In contrast to existing datasets,
  Qilin offers a comprehensive collection of user sessions with
  heterogeneous results like image-text notes, video notes, commercial notes, and
  direct answers, facilitating the development of advanced multimodal neural
  retrieval models across diverse task settings. To better model user
  satisfaction and support the analysis of heterogeneous user behaviors, we also
  collect extensive APP-level contextual signals and genuine user feedback.
  Notably, Qilin contains user-favored answers and their referred results for
  search requests triggering the Deep Query Answering (DQA) module. This allows
  not only the training \& evaluation of a Retrieval-augmented Generation (RAG)
  pipeline, but also the exploration of how such a module would affect users'
  search behavior. Through comprehensive analysis and experiments, we provide
  interesting findings and insights for further improving S\&R systems. We hope
  that Qilin will significantly contribute to the advancement of
  multimodal content platforms with S\&R services in the future.
paper_authors._id: 67c6a343ad6b7c2fa29d5e7e
paper_authors.name: Jia Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-04T16:08:10.744Z
paper_authors._id: 67c03221aed8409476d39da8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/67c03221aed8409476d39da8/eQIhOPRLNoiphsR145mfB.png
paper_authors.fullname: Jia Chen
paper_authors.type: user
paper_authors.user: Regulus309
paper_authors._id: 67c6a343ad6b7c2fa29d5e7f
paper_authors.name: Qian Dong
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-04T08:34:51.762Z
paper_authors._id: 60c0ed29d8bc072769d78f48
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60c0ed29d8bc072769d78f48/V6q6Tn4kzB46NIbTYw9pQ.jpeg
paper_authors.fullname: Qian Dong
paper_authors.type: user
paper_authors.user: qian
paper_authors._id: 67c6a343ad6b7c2fa29d5e80
paper_authors.name: Haitao Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-04T10:20:57.898Z
paper_authors._id: 67b5d91558369f6b38c5b596
paper_authors.avatarUrl: /avatars/18b08d5d9b05786cad34bc000c7606aa.svg
paper_authors.fullname: Haitao Li
paper_authors.type: user
paper_authors.user: haitaoli
paper_authors._id: 67c6a343ad6b7c2fa29d5e81
paper_authors.name: Xiaohui He
paper_authors._id: 67c6a343ad6b7c2fa29d5e82
paper_authors.name: Yan Gao
paper_authors._id: 67c6a343ad6b7c2fa29d5e83
paper_authors.name: Shaosheng Cao
paper_authors._id: 67c6a343ad6b7c2fa29d5e84
paper_authors.name: Yi Wu
paper_authors._id: 67c6a343ad6b7c2fa29d5e85
paper_authors.name: Ping Yang
paper_authors._id: 67c6a343ad6b7c2fa29d5e86
paper_authors.name: Chen Xu
paper_authors._id: 67c6a343ad6b7c2fa29d5e87
paper_authors.name: Yao Hu
paper_authors._id: 67c6a343ad6b7c2fa29d5e88
paper_authors.name: Qingyao Ai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-04T10:21:22.100Z
paper_authors._id: 6657e7045f6e35c7d541bdd8
paper_authors.avatarUrl: /avatars/368e5cef6c93543b2b92fbca79a4e4b9.svg
paper_authors.fullname: Qingyao Ai
paper_authors.type: user
paper_authors.user: aiqy
paper_authors._id: 67c6a343ad6b7c2fa29d5e89
paper_authors.name: Yiqun Liu
paper_title: Qilin: A Multimodal Information Retrieval Dataset with APP-level User
    Sessions
paper_summary: User-generated content (UGC) communities, especially those featuring
  multimodal content, improve user experiences by integrating visual and textual
  information into results (or items). The challenge of improving user
  experiences in complex systems with search and recommendation (S\&R) services
  has drawn significant attention from both academia and industry these years.
  However, the lack of high-quality datasets has limited the research progress on
  multimodal S\&R. To address the growing need for developing better S\&R
  services, we present a novel multimodal information retrieval dataset in this
  paper, namely Qilin. The dataset is collected from Xiaohongshu, a popular
  social platform with over 300 million monthly active users and an average
  search penetration rate of over 70\%. In contrast to existing datasets,
  Qilin offers a comprehensive collection of user sessions with
  heterogeneous results like image-text notes, video notes, commercial notes, and
  direct answers, facilitating the development of advanced multimodal neural
  retrieval models across diverse task settings. To better model user
  satisfaction and support the analysis of heterogeneous user behaviors, we also
  collect extensive APP-level contextual signals and genuine user feedback.
  Notably, Qilin contains user-favored answers and their referred results for
  search requests triggering the Deep Query Answering (DQA) module. This allows
  not only the training \& evaluation of a Retrieval-augmented Generation (RAG)
  pipeline, but also the exploration of how such a module would affect users'
  search behavior. Through comprehensive analysis and experiments, we provide
  interesting findings and insights for further improving S\&R systems. We hope
  that Qilin will significantly contribute to the advancement of
  multimodal content platforms with S\&R services in the future.
paper_ai_keywords: multimodal content search and recommendation (S\&R) multimodal information retrieval Qilin Xiaohongshu image-text notes video notes commercial notes direct answers multimodal neural retrieval models user sessions APP-level contextual signals genuine user feedback user-favored answers Deep Query Answering (DQA) Retrieval-augmented Generation (RAG)