title: MS MARCO Web Search: a Large-scale Information-rich Web Dataset with
    Millions of Real Click Labels
summary: Recent breakthroughs in large models have highlighted the critical
  significance of data scale, labels and modals. In this paper, we introduce MS
  MARCO Web Search, the first large-scale information-rich web dataset, featuring
  millions of real clicked query-document labels. This dataset closely mimics
  real-world web document and query distribution, provides rich information for
  various kinds of downstream tasks and encourages research in various areas,
  such as generic end-to-end neural indexer models, generic embedding models, and
  next generation information access system with large language models. MS MARCO
  Web Search offers a retrieval benchmark with three web retrieval challenge
  tasks that demand innovations in both machine learning and information
  retrieval system research domains. As the first dataset that meets large, real
  and rich data requirements, MS MARCO Web Search paves the way for future
  advancements in AI and system research. MS MARCO Web Search dataset is
  available at: https://github.com/microsoft/MS-MARCO-Web-Search.
paper_authors._id: 6642e998af62c6c266aa541f
paper_authors.name: Qi Chen
paper_authors._id: 6642e998af62c6c266aa5420
paper_authors.name: Xiubo Geng
paper_authors._id: 6642e998af62c6c266aa5421
paper_authors.name: Corby Rosset
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-14T08:28:10.235Z
paper_authors._id: 64ec43147e2ec711a761c594
paper_authors.avatarUrl: /avatars/e4fc3de141ff026eee6eaa0b91d34ff1.svg
paper_authors.fullname: corby
paper_authors.type: user
paper_authors.user: corbyrosset
paper_authors._id: 6642e998af62c6c266aa5422
paper_authors.name: Carolyn Buractaon
paper_authors._id: 6642e998af62c6c266aa5423
paper_authors.name: Jingwen Lu
paper_authors._id: 6642e998af62c6c266aa5424
paper_authors.name: Tao Shen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-14T08:29:10.835Z
paper_authors._id: 602331cdc1f3c79f98e4b378
paper_authors.avatarUrl: /avatars/fbcf4ac9059df87ee788dc431f46606a.svg
paper_authors.fullname: Tao Shen
paper_authors.type: user
paper_authors.user: taoshen58
paper_authors._id: 6642e998af62c6c266aa5425
paper_authors.name: Kun Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-23T08:40:35.767Z
paper_authors._id: 66f4fcbc29b10ae4c990a2e0
paper_authors.avatarUrl: /avatars/5d6df3aa5792a031c28274d428c46d84.svg
paper_authors.fullname: Kun Zhou
paper_authors.type: user
paper_authors.user: FrancisKunZhou
paper_authors._id: 6642e998af62c6c266aa5426
paper_authors.name: Chenyan Xiong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-14T08:30:29.074Z
paper_authors._id: 617ae49bc0ff6006217aa22e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1635543252231-617ae49bc0ff6006217aa22e.jpeg
paper_authors.fullname: Chenyan Xiong
paper_authors.type: user
paper_authors.user: xiongchenyan
paper_authors._id: 6642e998af62c6c266aa5427
paper_authors.name: Yeyun Gong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-14T08:30:57.752Z
paper_authors._id: 643f615aa16cd6d1f4c581de
paper_authors.avatarUrl: /avatars/47753a3e82b44f81881600c52e1e8495.svg
paper_authors.fullname: Yeyun Gong
paper_authors.type: user
paper_authors.user: yegong
paper_authors._id: 6642e998af62c6c266aa5428
paper_authors.name: Paul Bennett
paper_authors._id: 6642e998af62c6c266aa5429
paper_authors.name: Nick Craswell
paper_authors._id: 6642e998af62c6c266aa542a
paper_authors.name: Xing Xie
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-14T08:32:07.153Z
paper_authors._id: 64785ca82b3730b6b63b75a2
paper_authors.avatarUrl: /avatars/b0c4f7fb4d4c556118fa6b52d3615429.svg
paper_authors.fullname: Xing Xie
paper_authors.type: user
paper_authors.user: xingustc
paper_authors._id: 6642e998af62c6c266aa542b
paper_authors.name: Fan Yang
paper_authors._id: 6642e998af62c6c266aa542c
paper_authors.name: Bryan Tower
paper_authors._id: 6642e998af62c6c266aa542d
paper_authors.name: Nikhil Rao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-29T07:47:51.835Z
paper_authors._id: 68377cd94d2784008f519993
paper_authors.avatarUrl: /avatars/ea30ce393706e5167fe03b79ded94939.svg
paper_authors.fullname: Nikhil Rao
paper_authors.type: user
paper_authors.user: theclairvoyant
paper_authors._id: 6642e998af62c6c266aa542e
paper_authors.name: Anlei Dong
paper_authors._id: 6642e998af62c6c266aa542f
paper_authors.name: Wenqi Jiang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-14T08:33:16.464Z
paper_authors._id: 64004d45a3b8fe3ac0e8469d
paper_authors.avatarUrl: /avatars/d3dd9637e2dd725626c779d27905482c.svg
paper_authors.fullname: Wenqi Jiang
paper_authors.type: user
paper_authors.user: wenqij
paper_authors._id: 6642e998af62c6c266aa5430
paper_authors.name: Zheng Liu
paper_authors._id: 6642e998af62c6c266aa5431
paper_authors.name: Mingqin Li
paper_authors._id: 6642e998af62c6c266aa5432
paper_authors.name: Chuanjie Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-14T08:33:54.003Z
paper_authors._id: 6528b76d0b44378dcc54ec60
paper_authors.avatarUrl: /avatars/0943b4c7def02c1751e163aa758ccec5.svg
paper_authors.fullname: Chuanjie Liu
paper_authors.type: user
paper_authors.user: chuanjieliu
paper_authors._id: 6642e998af62c6c266aa5433
paper_authors.name: Zengzhong Li
paper_authors._id: 6642e998af62c6c266aa5434
paper_authors.name: Rangan Majumder
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-14T08:34:07.277Z
paper_authors._id: 64851c3610e6b874559ef7ae
paper_authors.avatarUrl: /avatars/2847eb17a5fa45c93280d4224c33ddf0.svg
paper_authors.fullname: Rangan Majumder
paper_authors.type: user
paper_authors.user: Rangosaur
paper_authors._id: 6642e998af62c6c266aa5435
paper_authors.name: Jennifer Neville
paper_authors._id: 6642e998af62c6c266aa5436
paper_authors.name: Andy Oakley
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-14T08:34:18.710Z
paper_authors._id: 630c42b0cd26ad7f60e047d5
paper_authors.avatarUrl: /avatars/2707066c2113ad03b24f12bf45b5fe35.svg
paper_authors.fullname: Andy Oakley
paper_authors.type: user
paper_authors.user: andyoakley
paper_authors._id: 6642e998af62c6c266aa5437
paper_authors.name: Knut Magne Risvik
paper_authors._id: 6642e998af62c6c266aa5438
paper_authors.name: Harsha Vardhan Simhadri
paper_authors._id: 6642e998af62c6c266aa5439
paper_authors.name: Manik Varma
paper_authors._id: 6642e998af62c6c266aa543a
paper_authors.name: Yujing Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-14T08:34:38.076Z
paper_authors._id: 6412e09e8c9885e10298d57f
paper_authors.avatarUrl: /avatars/0f748c80ba3cc3e8fcb40b284ec20b47.svg
paper_authors.fullname: yujingwang
paper_authors.type: user
paper_authors.user: wangyujing1234
paper_authors._id: 6642e998af62c6c266aa543b
paper_authors.name: Linjun Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-14T08:34:44.682Z
paper_authors._id: 641db563c9ce8fdbce4eaa86
paper_authors.avatarUrl: /avatars/ec4012d12a56cba5e27c9eaf58fc7691.svg
paper_authors.fullname: Linjun Yang
paper_authors.type: user
paper_authors.user: Wadeyang
paper_authors._id: 6642e998af62c6c266aa543c
paper_authors.name: Mao Yang
paper_authors._id: 6642e998af62c6c266aa543d
paper_authors.name: Ce Zhang
paper_title: MS MARCO Web Search: a Large-scale Information-rich Web Dataset with
    Millions of Real Click Labels
paper_summary: Recent breakthroughs in large models have highlighted the critical
  significance of data scale, labels and modals. In this paper, we introduce MS
  MARCO Web Search, the first large-scale information-rich web dataset, featuring
  millions of real clicked query-document labels. This dataset closely mimics
  real-world web document and query distribution, provides rich information for
  various kinds of downstream tasks and encourages research in various areas,
  such as generic end-to-end neural indexer models, generic embedding models, and
  next generation information access system with large language models. MS MARCO
  Web Search offers a retrieval benchmark with three web retrieval challenge
  tasks that demand innovations in both machine learning and information
  retrieval system research domains. As the first dataset that meets large, real
  and rich data requirements, MS MARCO Web Search paves the way for future
  advancements in AI and system research. MS MARCO Web Search dataset is
  available at: https://github.com/microsoft/MS-MARCO-Web-Search.
paper_ai_summary: MS MARCO Web Search provides a large-scale, information-rich web dataset with real-world query-document labels for training and evaluating neural indexer models, embedding models, and information access systems.
paper_ai_keywords: generic end-to-end neural indexer models generic embedding models next generation information access system large language models retrieval benchmark web retrieval challenge tasks machine learning information retrieval system research