title: OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training
    Tokens
summary: We present OLMoTrace, the first system that traces the outputs of language
  models back to their full, multi-trillion-token training data in real time.
  OLMoTrace finds and shows verbatim matches between segments of language model
  output and documents in the training text corpora. Powered by an extended
  version of infini-gram (Liu et al., 2024), our system returns tracing results
  within a few seconds. OLMoTrace can help users understand the behavior of
  language models through the lens of their training data. We showcase how it can
  be used to explore fact checking, hallucination, and the creativity of language
  models. OLMoTrace is publicly available and fully open-source.
paper_authors._id: 67f72bb1f9d51b79dca06d0a
paper_authors.name: Jiacheng Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-10T06:39:44.913Z
paper_authors._id: 635f46d1928a42bc95cfcf7c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/635f46d1928a42bc95cfcf7c/5KF8aLiDCJdl7B1SdJ-7V.png
paper_authors.fullname: Jiacheng Liu
paper_authors.type: user
paper_authors.user: liujch1998
paper_authors._id: 67f72bb1f9d51b79dca06d0b
paper_authors.name: Taylor Blanton
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-10T08:02:05.681Z
paper_authors._id: 6675a65557208377a15f745b
paper_authors.avatarUrl: /avatars/361dc6d0919f4d4545ff4fdd005332b5.svg
paper_authors.fullname: Taylor Blanton
paper_authors.type: user
paper_authors.user: taylorb
paper_authors._id: 67f72bb1f9d51b79dca06d0c
paper_authors.name: Yanai Elazar
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-10T08:02:12.016Z
paper_authors._id: 623ca115a795593324c4353f
paper_authors.avatarUrl: /avatars/bf11fe728df2786d52ed4d2de12b48d3.svg
paper_authors.fullname: Yanai Elazar
paper_authors.type: user
paper_authors.user: yanaiela
paper_authors._id: 67f72bb1f9d51b79dca06d0d
paper_authors.name: Sewon Min
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-10T08:02:17.909Z
paper_authors._id: 63a76d0de27a6dbd485fe863
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63a76d0de27a6dbd485fe863/qJJwHOuvyQGq1o0KscOF_.jpeg
paper_authors.fullname: Sewon Min
paper_authors.type: user
paper_authors.user: sewon
paper_authors._id: 67f72bb1f9d51b79dca06d0e
paper_authors.name: YenSung Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-10T08:02:26.950Z
paper_authors._id: 6697093a37d2483826562c24
paper_authors.avatarUrl: /avatars/0e526b4be6db07e2485f7ef862080339.svg
paper_authors.fullname: Chen
paper_authors.type: user
paper_authors.user: Yensung
paper_authors._id: 67f72bb1f9d51b79dca06d0f
paper_authors.name: Arnavi Chheda-Kothary
paper_authors._id: 67f72bb1f9d51b79dca06d10
paper_authors.name: Huy Tran
paper_authors._id: 67f72bb1f9d51b79dca06d11
paper_authors.name: Byron Bischoff
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-11T07:25:58.346Z
paper_authors._id: 67f85d757f56304dc72b4961
paper_authors.avatarUrl: /avatars/6ae806d25632b250220145feae200058.svg
paper_authors.fullname: Byron Bischoff
paper_authors.type: user
paper_authors.user: byronbischoff
paper_authors._id: 67f72bb1f9d51b79dca06d12
paper_authors.name: Eric Marsh
paper_authors._id: 67f72bb1f9d51b79dca06d13
paper_authors.name: Michael Schmitz
paper_authors._id: 67f72bb1f9d51b79dca06d14
paper_authors.name: Cassidy Trier
paper_authors._id: 67f72bb1f9d51b79dca06d15
paper_authors.name: Aaron Sarnat
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-10T08:03:34.212Z
paper_authors._id: 65b1520bf7638a13a641a620
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65b1520bf7638a13a641a620/KTauZL0kXlmYnbkI2lFBG.png
paper_authors.fullname: Aaron Sarnat
paper_authors.type: user
paper_authors.user: aaronsarnat
paper_authors._id: 67f72bb1f9d51b79dca06d16
paper_authors.name: Jenna James
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-11T07:26:00.170Z
paper_authors._id: 67f7fb63ef0ad511d684f9fc
paper_authors.avatarUrl: /avatars/b49d723fe735f71d7f6b809813e4ccc5.svg
paper_authors.fullname: James James
paper_authors.type: user
paper_authors.user: jennajames
paper_authors._id: 67f72bb1f9d51b79dca06d17
paper_authors.name: Jon Borchardt
paper_authors._id: 67f72bb1f9d51b79dca06d18
paper_authors.name: Bailey Kuehl
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-10T08:03:47.506Z
paper_authors._id: 65316953791d5a2611426c20
paper_authors.avatarUrl: /avatars/e632a9a30a57f62d59f9fe42eba8fd7d.svg
paper_authors.fullname: bailey kuehl
paper_authors.type: user
paper_authors.user: baileyk
paper_authors._id: 67f72bb1f9d51b79dca06d19
paper_authors.name: Evie Cheng
paper_authors._id: 67f72bb1f9d51b79dca06d1a
paper_authors.name: Karen Farley
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-10T08:03:58.149Z
paper_authors._id: 66213c05e288b64070184cac
paper_authors.avatarUrl: /avatars/ded6a173e60722200b372b8b046fc359.svg
paper_authors.fullname: Karen Farley
paper_authors.type: user
paper_authors.user: AI2Karen
paper_authors._id: 67f72bb1f9d51b79dca06d1b
paper_authors.name: Sruthi Sreeram
paper_authors._id: 67f72bb1f9d51b79dca06d1c
paper_authors.name: Taira Anderson
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-10T08:04:09.193Z
paper_authors._id: 65de20ad4e73a7dea7fb4f08
paper_authors.avatarUrl: /avatars/f3b0ad6cc9417e8ea3f0607fa62824d1.svg
paper_authors.fullname: Taira Anderson
paper_authors.type: user
paper_authors.user: tairaa
paper_authors._id: 67f72bb1f9d51b79dca06d1d
paper_authors.name: David Albright
paper_authors._id: 67f72bb1f9d51b79dca06d1e
paper_authors.name: Carissa Schoenick
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-10T08:04:25.492Z
paper_authors._id: 6024546dc1f3c79f98e4b384
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1612993792778-6024546dc1f3c79f98e4b384.jpeg
paper_authors.fullname: Carissa Schoenick
paper_authors.type: user
paper_authors.user: CarissaS
paper_authors._id: 67f72bb1f9d51b79dca06d1f
paper_authors.name: Luca Soldaini
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-10T08:04:31.958Z
paper_authors._id: 5f04d8c45d08220171a0ad32
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/5f04d8c45d08220171a0ad32/uXEta6nqBabrUlAOXnS5g.jpeg
paper_authors.fullname: Luca Soldaini
paper_authors.type: user
paper_authors.user: soldni
paper_authors._id: 67f72bb1f9d51b79dca06d20
paper_authors.name: Dirk Groeneveld
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-10T08:04:40.029Z
paper_authors._id: 60369745413a78f892e7339c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1636671879171-60369745413a78f892e7339c.png
paper_authors.fullname: Dirk Groeneveld
paper_authors.type: user
paper_authors.user: dirkgr
paper_authors._id: 67f72bb1f9d51b79dca06d21
paper_authors.name: Rock Yuren Pang
paper_authors._id: 67f72bb1f9d51b79dca06d22
paper_authors.name: Pang Wei Koh
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-10T08:04:53.298Z
paper_authors._id: 641b4263abfce26bcf7b27de
paper_authors.avatarUrl: /avatars/e91b4205e4f74b0dd8c333c23203a924.svg
paper_authors.fullname: Pang Wei Koh
paper_authors.type: user
paper_authors.user: pangwei
paper_authors._id: 67f72bb1f9d51b79dca06d23
paper_authors.name: Noah A. Smith
paper_authors._id: 67f72bb1f9d51b79dca06d24
paper_authors.name: Sophie Lebrecht
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-10T08:05:06.279Z
paper_authors._id: 65b301f04c9e50e74a893954
paper_authors.avatarUrl: /avatars/f52366959f9e7613576603c0272ff2c5.svg
paper_authors.fullname: Sophie Lebrecht
paper_authors.type: user
paper_authors.user: Lebrechts
paper_authors._id: 67f72bb1f9d51b79dca06d25
paper_authors.name: Yejin Choi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-10T08:05:13.983Z
paper_authors._id: 64d42729f63b01b7f676b176
paper_authors.avatarUrl: /avatars/52e54bdd6a1fb6c774a40cd70f3d7925.svg
paper_authors.fullname: Yejin Choi
paper_authors.type: user
paper_authors.user: yejinchoinka
paper_authors._id: 67f72bb1f9d51b79dca06d26
paper_authors.name: Hannaneh Hajishirzi
paper_authors._id: 67f72bb1f9d51b79dca06d27
paper_authors.name: Ali Farhadi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-10T08:05:24.948Z
paper_authors._id: 6660d4c1818c5c5ca0f31266
paper_authors.avatarUrl: /avatars/1d2972894cb3b9df1900fdb162d9c364.svg
paper_authors.fullname: alifarhadi 
paper_authors.type: user
paper_authors.user: alifarhadi051
paper_authors._id: 67f72bb1f9d51b79dca06d28
paper_authors.name: Jesse Dodge
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-10T08:05:31.942Z
paper_authors._id: 6283f38567d336d3e5d5280e
paper_authors.avatarUrl: /avatars/d0a54aaec74a90b050e671c191b87a80.svg
paper_authors.fullname: Jesse Dodge
paper_authors.type: user
paper_authors.user: JesseDodge
paper_title: OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training
    Tokens
paper_summary: We present OLMoTrace, the first system that traces the outputs of language
  models back to their full, multi-trillion-token training data in real time.
  OLMoTrace finds and shows verbatim matches between segments of language model
  output and documents in the training text corpora. Powered by an extended
  version of infini-gram (Liu et al., 2024), our system returns tracing results
  within a few seconds. OLMoTrace can help users understand the behavior of
  language models through the lens of their training data. We showcase how it can
  be used to explore fact checking, hallucination, and the creativity of language
  models. OLMoTrace is publicly available and fully open-source.
paper_ai_summary: OLMoTrace traces language model outputs to their training data in real-time, providing insights into fact checking, hallucination, and creativity through verbatim matches and infini-gram technology.
paper_ai_keywords: infini-gram