title: OLMo: Accelerating the Science of Language Models
summary: Language models (LMs) have become ubiquitous in both NLP research and in
  commercial product offerings. As their commercial importance has surged, the
  most powerful models have become closed off, gated behind proprietary
  interfaces, with important details of their training data, architectures, and
  development undisclosed. Given the importance of these details in
  scientifically studying these models, including their biases and potential
  risks, we believe it is essential for the research community to have access to
  powerful, truly open LMs. To this end, this technical report details the first
  release of OLMo, a state-of-the-art, truly Open Language Model and its
  framework to build and study the science of language modeling. Unlike most
  prior efforts that have only released model weights and inference code, we
  release OLMo and the whole framework, including training data and training and
  evaluation code. We hope this release will empower and strengthen the open
  research community and inspire a new wave of innovation.
paper_authors._id: 65bc696d7c6ad1994b7fb464
paper_authors.name: Dirk Groeneveld
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T09:22:31.336Z
paper_authors._id: 60369745413a78f892e7339c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1636671879171-60369745413a78f892e7339c.png
paper_authors.fullname: Dirk Groeneveld
paper_authors.type: user
paper_authors.user: dirkgr
paper_authors._id: 65bc696d7c6ad1994b7fb465
paper_authors.name: Iz Beltagy
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T09:22:39.212Z
paper_authors._id: 5ec07694ed25d76864d553f4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1589816744885-5ec07694ed25d76864d553f4.jpeg
paper_authors.fullname: Iz Beltagy
paper_authors.type: user
paper_authors.user: beltagy
paper_authors._id: 65bc696d7c6ad1994b7fb466
paper_authors.name: Pete Walsh
paper_authors._id: 65bc696d7c6ad1994b7fb467
paper_authors.name: Akshita Bhagia
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T09:22:59.053Z
paper_authors._id: 6282937a9624ab5f75c977e1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6282937a9624ab5f75c977e1/ZOysQbGzEzmrN_SaTQgOP.jpeg
paper_authors.fullname: Akshita Bhagia
paper_authors.type: user
paper_authors.user: akshitab
paper_authors._id: 65bc696d7c6ad1994b7fb468
paper_authors.name: Rodney Kinney
paper_authors._id: 65bc696d7c6ad1994b7fb469
paper_authors.name: Oyvind Tafjord
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T09:23:19.403Z
paper_authors._id: 609b1aeff7c1303cfd1760f3
paper_authors.avatarUrl: /avatars/3cb448514885656466886d7c84e12af1.svg
paper_authors.fullname: Oyvind Tafjord
paper_authors.type: user
paper_authors.user: OyvindTafjord
paper_authors._id: 65bc696d7c6ad1994b7fb46a
paper_authors.name: Ananya Harsh Jha
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T09:29:37.268Z
paper_authors._id: 6582bee321aa786b76beba44
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6582bee321aa786b76beba44/1YqJBHmLxMNRaGtU-lbKh.jpeg
paper_authors.fullname: Ananya Harsh Jha
paper_authors.type: user
paper_authors.user: ananyahjha93
paper_authors._id: 65bc696d7c6ad1994b7fb46b
paper_authors.name: Hamish Ivison
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T09:30:01.429Z
paper_authors._id: 62608fc2ffe8827cb1d89f9f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1654027835241-62608fc2ffe8827cb1d89f9f.png
paper_authors.fullname: Hamish Ivison
paper_authors.type: user
paper_authors.user: hamishivi
paper_authors._id: 65bc696d7c6ad1994b7fb46c
paper_authors.name: Ian Magnusson
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T09:30:12.254Z
paper_authors._id: 62bddd0b1e22ec8427a0f27e
paper_authors.avatarUrl: /avatars/4528b10d6b94bdbee2fe3c2d28331945.svg
paper_authors.fullname: Ian Magnusson
paper_authors.type: user
paper_authors.user: IanMagnusson
paper_authors._id: 65bc696d7c6ad1994b7fb46d
paper_authors.name: Yizhong Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-02T09:00:37.294Z
paper_authors._id: 6269be67d1ac0cde592aba29
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6269be67d1ac0cde592aba29/EP3zZbD7-jWK9ITvHGbnZ.jpeg
paper_authors.fullname: Yizhong Wang
paper_authors.type: user
paper_authors.user: yizhongw
paper_authors._id: 65bc696d7c6ad1994b7fb46e
paper_authors.name: Shane Arora
paper_authors._id: 65bc696d7c6ad1994b7fb46f
paper_authors.name: David Atkinson
paper_authors._id: 65bc696d7c6ad1994b7fb470
paper_authors.name: Russell Authur
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T09:30:52.208Z
paper_authors._id: 62712c97bcef985363d9e368
paper_authors.avatarUrl: /avatars/0fcc2f096bc6ba50a43cf1345d5c0066.svg
paper_authors.fullname: Russell Authur
paper_authors.type: user
paper_authors.user: rauthur
paper_authors._id: 65bc696d7c6ad1994b7fb471
paper_authors.name: Khyathi Raghavi Chandu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T09:31:01.103Z
paper_authors._id: 62ebe6472f8999993a847bb6
paper_authors.avatarUrl: /avatars/29fb590f8116bdc9c55f7a333e2c4280.svg
paper_authors.fullname: Khyathi Raghavi Chandu
paper_authors.type: user
paper_authors.user: khyathi
paper_authors._id: 65bc696d7c6ad1994b7fb472
paper_authors.name: Arman Cohan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T09:31:07.535Z
paper_authors._id: 5f5ba21188f57f65f951f255
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1599840760465-noauth.png
paper_authors.fullname: Arman Cohan
paper_authors.type: user
paper_authors.user: armanc
paper_authors._id: 65bc696d7c6ad1994b7fb473
paper_authors.name: Jennifer Dumas
paper_authors._id: 65bc696d7c6ad1994b7fb474
paper_authors.name: Yanai Elazar
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T09:31:23.561Z
paper_authors._id: 623ca115a795593324c4353f
paper_authors.avatarUrl: /avatars/bf11fe728df2786d52ed4d2de12b48d3.svg
paper_authors.fullname: Yanai Elazar
paper_authors.type: user
paper_authors.user: yanaiela
paper_authors._id: 65bc696d7c6ad1994b7fb475
paper_authors.name: Yuling Gu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T11:10:37.473Z
paper_authors._id: 62a8b91c1277a49db0c5c080
paper_authors.avatarUrl: /avatars/9cf834a02b67af862659c9b9d4367a52.svg
paper_authors.fullname: Yuling Gu
paper_authors.type: user
paper_authors.user: SaveBertAndGpt
paper_authors._id: 65bc696d7c6ad1994b7fb476
paper_authors.name: Jack Hessel
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T11:10:46.037Z
paper_authors._id: 625498644f4edf771516b2cb
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1664497251391-625498644f4edf771516b2cb.jpeg
paper_authors.fullname: Jack Hessel
paper_authors.type: user
paper_authors.user: jmhessel
paper_authors._id: 65bc696d7c6ad1994b7fb477
paper_authors.name: Tushar Khot
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T11:10:52.780Z
paper_authors._id: 62cdf2dbaac2c91c95581830
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62cdf2dbaac2c91c95581830/N3NuO7j53qkauJOf3kkRS.jpeg
paper_authors.fullname: Tushar Khot
paper_authors.type: user
paper_authors.user: tusharkhot
paper_authors._id: 65bc696d7c6ad1994b7fb478
paper_authors.name: William Merrill
paper_authors._id: 65bc696d7c6ad1994b7fb479
paper_authors.name: Jacob Morrison
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T11:11:13.650Z
paper_authors._id: 63f51f6cb51da4d61da72937
paper_authors.avatarUrl: /avatars/6023ecaafb28b7599f59816381d58bd0.svg
paper_authors.fullname: Jacob Morrison
paper_authors.type: user
paper_authors.user: jacobmorrison
paper_authors._id: 65bc696d7c6ad1994b7fb47a
paper_authors.name: Niklas Muennighoff
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T11:11:23.013Z
paper_authors._id: 5f1eb362eec0ad2a071ad6e2
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/5f1eb362eec0ad2a071ad6e2/IXMYkYKuTwn6kBdWnQeeY.png
paper_authors.fullname: Niklas Muennighoff
paper_authors.type: user
paper_authors.user: Muennighoff
paper_authors._id: 65bc696d7c6ad1994b7fb47b
paper_authors.name: Aakanksha Naik
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T11:11:51.673Z
paper_authors._id: 6268981926476157a9262a73
paper_authors.avatarUrl: /avatars/a6e4d81580039e8680c194d89a716535.svg
paper_authors.fullname: Aakanksha Naik
paper_authors.type: user
paper_authors.user: aakanksha
paper_authors._id: 65bc696d7c6ad1994b7fb47c
paper_authors.name: Crystal Nam
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T11:12:00.866Z
paper_authors._id: 64a7264f44387a55046f92e5
paper_authors.avatarUrl: /avatars/7f6e076a2a535957bc9c7ac4aeae716d.svg
paper_authors.fullname: Crystal Nam
paper_authors.type: user
paper_authors.user: CrystalN
paper_authors._id: 65bc696d7c6ad1994b7fb47d
paper_authors.name: Matthew E. Peters
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T11:12:35.568Z
paper_authors._id: 63d0455a8cf6c8e4d2087eeb
paper_authors.avatarUrl: /avatars/e86cb0ffbfa3c64f162945c40a2fdb89.svg
paper_authors.fullname: Matthew Peters
paper_authors.type: user
paper_authors.user: matt-peters
paper_authors._id: 65bc696d7c6ad1994b7fb47e
paper_authors.name: Valentina Pyatkin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T11:12:48.140Z
paper_authors._id: 6556cff80e7a7067a934445f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6556cff80e7a7067a934445f/PoT7qQ6tVqLGGrYvGBbr8.jpeg
paper_authors.fullname: Valentina Pyatkin
paper_authors.type: user
paper_authors.user: valpy
paper_authors._id: 65bc696d7c6ad1994b7fb47f
paper_authors.name: Abhilasha Ravichander
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T11:12:55.122Z
paper_authors._id: 6349886c429608888c42319a
paper_authors.avatarUrl: /avatars/f84b5fe8b76172878274754e3399d6ec.svg
paper_authors.fullname: Abhilasha Ravichander
paper_authors.type: user
paper_authors.user: lasha-nlp
paper_authors._id: 65bc696d7c6ad1994b7fb480
paper_authors.name: Dustin Schwenk
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T11:13:26.572Z
paper_authors._id: 647e791b7f9ad5e44babb965
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/647e791b7f9ad5e44babb965/Vjfy4c3ooDwU24N7mvRdy.jpeg
paper_authors.fullname: Dustin Schwenk
paper_authors.type: user
paper_authors.user: drschwenk
paper_authors._id: 65bc696d7c6ad1994b7fb481
paper_authors.name: Saurabh Shah
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T11:13:47.158Z
paper_authors._id: 65244773e6e5f6b1033bb371
paper_authors.avatarUrl: /avatars/2366893a2c6503c3894be5c427d9cbbe.svg
paper_authors.fullname: Saurabh Shah
paper_authors.type: user
paper_authors.user: saurabh-shah
paper_authors._id: 65bc696d7c6ad1994b7fb482
paper_authors.name: Will Smith
paper_authors._id: 65bc696d7c6ad1994b7fb483
paper_authors.name: Emma Strubell
paper_authors._id: 65bc696d7c6ad1994b7fb484
paper_authors.name: Nishant Subramani
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T11:14:13.026Z
paper_authors._id: 617b167b941993035fbaf2bf
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/617b167b941993035fbaf2bf/Wt3QMbjX9EMxdIrQdrF2l.jpeg
paper_authors.fullname: Nishant Subramani
paper_authors.type: user
paper_authors.user: nsubramani23
paper_authors._id: 65bc696d7c6ad1994b7fb485
paper_authors.name: Mitchell Wortsman
paper_authors._id: 65bc696d7c6ad1994b7fb486
paper_authors.name: Pradeep Dasigi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T11:16:11.327Z
paper_authors._id: 6408fcc93461c51cf735a61e
paper_authors.avatarUrl: /avatars/619f3653911d111f046a5a6c30fc8319.svg
paper_authors.fullname: Pradeep Dasigi
paper_authors.type: user
paper_authors.user: pradeepd
paper_authors._id: 65bc696d7c6ad1994b7fb487
paper_authors.name: Nathan Lambert
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T11:16:29.665Z
paper_authors._id: 628e5f90a9a3c754c1f7c88f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/628e5f90a9a3c754c1f7c88f/iWqMY_l6dalrgRaJZWbK3.png
paper_authors.fullname: Nathan Lambert
paper_authors.type: user
paper_authors.user: natolambert
paper_authors._id: 65bc696d7c6ad1994b7fb488
paper_authors.name: Kyle Richardson
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-02T09:00:39.435Z
paper_authors._id: 659c6f5e1d398a238152227d
paper_authors.avatarUrl: /avatars/dba97fe8cb825102f1eae97104a71f64.svg
paper_authors.fullname: Kyle Richardson
paper_authors.type: user
paper_authors.user: yakazimir
paper_authors._id: 65bc696d7c6ad1994b7fb489
paper_authors.name: Luke Zettlemoyer
paper_authors._id: 65bc696d7c6ad1994b7fb48a
paper_authors.name: Jesse Dodge
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T11:16:38.638Z
paper_authors._id: 6283f38567d336d3e5d5280e
paper_authors.avatarUrl: /avatars/d0a54aaec74a90b050e671c191b87a80.svg
paper_authors.fullname: Jesse Dodge
paper_authors.type: user
paper_authors.user: JesseDodge
paper_authors._id: 65bc696d7c6ad1994b7fb48b
paper_authors.name: Kyle Lo
paper_authors._id: 65bc696d7c6ad1994b7fb48c
paper_authors.name: Luca Soldaini
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-02T11:16:51.964Z
paper_authors._id: 5f04d8c45d08220171a0ad32
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/5f04d8c45d08220171a0ad32/uXEta6nqBabrUlAOXnS5g.jpeg
paper_authors.fullname: Luca Soldaini
paper_authors.type: user
paper_authors.user: soldni
paper_authors._id: 65bc696d7c6ad1994b7fb48d
paper_authors.name: Noah A. Smith
paper_authors._id: 65bc696d7c6ad1994b7fb48e
paper_authors.name: Hannaneh Hajishirzi
paper_title: OLMo: Accelerating the Science of Language Models
paper_summary: Language models (LMs) have become ubiquitous in both NLP research and in
  commercial product offerings. As their commercial importance has surged, the
  most powerful models have become closed off, gated behind proprietary
  interfaces, with important details of their training data, architectures, and
  development undisclosed. Given the importance of these details in
  scientifically studying these models, including their biases and potential
  risks, we believe it is essential for the research community to have access to
  powerful, truly open LMs. To this end, this technical report details the first
  release of OLMo, a state-of-the-art, truly Open Language Model and its
  framework to build and study the science of language modeling. Unlike most
  prior efforts that have only released model weights and inference code, we
  release OLMo and the whole framework, including training data and training and
  evaluation code. We hope this release will empower and strengthen the open
  research community and inspire a new wave of innovation.
paper_ai_keywords: Language models NLP research training data model architectures development biases potential risks truly open LMs framework to build and study the science of language modeling training and evaluation code open research community