title: LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset
summary: Studying how people interact with large language models (LLMs) in real-world
  scenarios is increasingly important due to their widespread use in various
  applications. In this paper, we introduce LMSYS-Chat-1M, a large-scale dataset
  containing one million real-world conversations with 25 state-of-the-art LLMs.
  This dataset is collected from 210K unique IP addresses in the wild on our
  Vicuna demo and Chatbot Arena website. We offer an overview of the dataset's
  content, including its curation process, basic statistics, and topic
  distribution, highlighting its diversity, originality, and scale. We
  demonstrate its versatility through four use cases: developing content
  moderation models that perform similarly to GPT-4, building a safety benchmark,
  training instruction-following models that perform similarly to Vicuna, and
  creating challenging benchmark questions. We believe that this dataset will
  serve as a valuable resource for understanding and advancing LLM capabilities.
  The dataset is publicly available at
  https://huggingface.co/datasets/lmsys/lmsys-chat-1m.
paper_authors._id: 650ce0c54d77a0952255a311
paper_authors.name: Lianmin Zheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-22T15:08:26.491Z
paper_authors._id: 62d35f3ceaf3858ce253ab7a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62d35f3ceaf3858ce253ab7a/3yiiMriltXJBi242FZY-W.jpeg
paper_authors.fullname: Lianmin
paper_authors.type: user
paper_authors.user: lmzheng
paper_authors._id: 650ce0c54d77a0952255a312
paper_authors.name: Wei-Lin Chiang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-22T10:10:19.030Z
paper_authors._id: 63071772fd79b417f1b92291
paper_authors.avatarUrl: /avatars/ec68d20dcedf4553bb31d9f6e0ded813.svg
paper_authors.fullname: Wei-Lin Chiang
paper_authors.type: user
paper_authors.user: weichiang
paper_authors._id: 650ce0c54d77a0952255a313
paper_authors.name: Ying Sheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-22T15:03:21.201Z
paper_authors._id: 6455b969654d8bccae50736c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6455b969654d8bccae50736c/WsK0_sbQSVeosi-fh-O_U.jpeg
paper_authors.fullname: Ying Sheng
paper_authors.type: user
paper_authors.user: ying1123
paper_authors._id: 650ce0c54d77a0952255a314
paper_authors.name: Tianle Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-09-22T15:03:30.024Z
paper_authors._id: 64ab7f188a1a9187c219e597
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64ab7f188a1a9187c219e597/cieU4bdmPIn9dTNIVf3n7.jpeg
paper_authors.fullname: Tianle Li
paper_authors.type: user
paper_authors.user: Timmli
paper_authors._id: 650ce0c54d77a0952255a315
paper_authors.name: Siyuan Zhuang
paper_authors._id: 650ce0c54d77a0952255a316
paper_authors.name: Zhanghao Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-22T15:03:54.858Z
paper_authors._id: 6429b84e8852afdf89beeae5
paper_authors.avatarUrl: /avatars/7cfc1a484077a8d511c46392de8dee07.svg
paper_authors.fullname: Zhanghao Wu
paper_authors.type: user
paper_authors.user: michaelvll
paper_authors._id: 650ce0c54d77a0952255a317
paper_authors.name: Yonghao Zhuang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-22T15:04:02.791Z
paper_authors._id: 64ab987073790912c7ad79e6
paper_authors.avatarUrl: /avatars/bfa00418c46da0fd1e522d6561944ec9.svg
paper_authors.fullname: Yonghao Zhuang
paper_authors.type: user
paper_authors.user: ZYHowell
paper_authors._id: 650ce0c54d77a0952255a318
paper_authors.name: Zhuohan Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-22T15:04:10.259Z
paper_authors._id: 649dd1fae16abbd8fd7a3bdf
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/2t1p0Ku7qmNcSapRdzxv9.jpeg
paper_authors.fullname: Zhuohan  Li
paper_authors.type: user
paper_authors.user: zhuohan123
paper_authors._id: 650ce0c54d77a0952255a319
paper_authors.name: Zi Lin
paper_authors._id: 650ce0c54d77a0952255a31a
paper_authors.name: Eric. P Xing
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-22T15:05:00.730Z
paper_authors._id: 64ff67722ad36636be6c4542
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/sLIrNelAWPVOy4e3oo5LB.jpeg
paper_authors.fullname: Eric Xing
paper_authors.type: user
paper_authors.user: EricX003
paper_authors._id: 650ce0c54d77a0952255a31b
paper_authors.name: Joseph E. Gonzalez
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-22T15:06:07.969Z
paper_authors._id: 645d2e8401f4eaab2a0878ce
paper_authors.avatarUrl: /avatars/1273c5fb607b4b622a746a42692fa632.svg
paper_authors.fullname: Joseph E. Gonzalez
paper_authors.type: user
paper_authors.user: ProfJoeyG
paper_authors._id: 650ce0c54d77a0952255a31c
paper_authors.name: Ion Stoica
paper_authors._id: 650ce0c54d77a0952255a31d
paper_authors.name: Hao Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-22T15:09:17.704Z
paper_authors._id: 62d363143eebd640a4fa41fa
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62d363143eebd640a4fa41fa/pvPwXlJ5OOb-UIfmffv4E.jpeg
paper_authors.fullname: Hao Zhang
paper_authors.type: user
paper_authors.user: zhisbug
paper_title: LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset
paper_summary: Studying how people interact with large language models (LLMs) in real-world
  scenarios is increasingly important due to their widespread use in various
  applications. In this paper, we introduce LMSYS-Chat-1M, a large-scale dataset
  containing one million real-world conversations with 25 state-of-the-art LLMs.
  This dataset is collected from 210K unique IP addresses in the wild on our
  Vicuna demo and Chatbot Arena website. We offer an overview of the dataset's
  content, including its curation process, basic statistics, and topic
  distribution, highlighting its diversity, originality, and scale. We
  demonstrate its versatility through four use cases: developing content
  moderation models that perform similarly to GPT-4, building a safety benchmark,
  training instruction-following models that perform similarly to Vicuna, and
  creating challenging benchmark questions. We believe that this dataset will
  serve as a valuable resource for understanding and advancing LLM capabilities.
  The dataset is publicly available at
  https://huggingface.co/datasets/lmsys/lmsys-chat-1m.
paper_ai_summary: A large-scale dataset of one million real-world conversations with state-of-the-art LLMs provides a variety of applications including content moderation, safety benchmarking, and instruction-following model training.
paper_ai_keywords: large language models LLMs Vicuna demo Chatbot Arena content moderation models safety benchmark instruction-following models