OLMo: Accelerating the Science of Language Models
Language models (LMs) have become ubiquitous in both NLP research and in
commercial product offerings. As their commercial importance has surged, the
most powerful models have become closed off, gated behind proprietary
interfaces, with important details of their training data, architectures, and
development undisclosed. Given the importance of these details in
scientifically studying these models, including their biases and potential
risks, we believe it is essential for the research community to have access to
powerful, truly open LMs. To this end, this technical report details the first
release of OLMo, a state-of-the-art, truly Open Language Model and its
framework to build and study the science of language modeling. Unlike most
prior efforts that have only released model weights and inference code, we
release OLMo and the whole framework, including training data and training and
evaluation code. We hope this release will empower and strengthen the open
research community and inspire a new wave of innovation.
65bc696d7c6ad1994b7fb464
Dirk Groeneveld
admin_assigned
2024-02-02T09:22:31.336Z
60369745413a78f892e7339c
https://cdn-avatars.huggingface.co/v1/production/uploads/1636671879171-60369745413a78f892e7339c.png
Dirk Groeneveld
user
dirkgr
65bc696d7c6ad1994b7fb465
Iz Beltagy
admin_assigned
2024-02-02T09:22:39.212Z
5ec07694ed25d76864d553f4
https://cdn-avatars.huggingface.co/v1/production/uploads/1589816744885-5ec07694ed25d76864d553f4.jpeg
Iz Beltagy
user
beltagy
65bc696d7c6ad1994b7fb466
Pete Walsh
65bc696d7c6ad1994b7fb467
Akshita Bhagia
admin_assigned
2024-02-02T09:22:59.053Z
6282937a9624ab5f75c977e1
https://cdn-avatars.huggingface.co/v1/production/uploads/6282937a9624ab5f75c977e1/ZOysQbGzEzmrN_SaTQgOP.jpeg
Akshita Bhagia
user
akshitab
65bc696d7c6ad1994b7fb468
Rodney Kinney
65bc696d7c6ad1994b7fb469
Oyvind Tafjord
admin_assigned
2024-02-02T09:23:19.403Z
609b1aeff7c1303cfd1760f3
/avatars/3cb448514885656466886d7c84e12af1.svg
Oyvind Tafjord
user
OyvindTafjord
65bc696d7c6ad1994b7fb46a
Ananya Harsh Jha
admin_assigned
2024-02-02T09:29:37.268Z
6582bee321aa786b76beba44
https://cdn-avatars.huggingface.co/v1/production/uploads/6582bee321aa786b76beba44/1YqJBHmLxMNRaGtU-lbKh.jpeg
Ananya Harsh Jha
user
ananyahjha93
65bc696d7c6ad1994b7fb46b
Hamish Ivison
admin_assigned
2024-02-02T09:30:01.429Z
62608fc2ffe8827cb1d89f9f
https://cdn-avatars.huggingface.co/v1/production/uploads/1654027835241-62608fc2ffe8827cb1d89f9f.png
Hamish Ivison
user
hamishivi
65bc696d7c6ad1994b7fb46c
Ian Magnusson
admin_assigned
2024-02-02T09:30:12.254Z
62bddd0b1e22ec8427a0f27e
/avatars/c433d0fb2945c70d1998f1e0e98d265f.svg
Ian Magnusson
user
IanMagnusson
65bc696d7c6ad1994b7fb46d
Yizhong Wang
claimed_verified
2024-02-02T09:00:37.294Z
6269be67d1ac0cde592aba29
https://cdn-avatars.huggingface.co/v1/production/uploads/6269be67d1ac0cde592aba29/EP3zZbD7-jWK9ITvHGbnZ.jpeg
Yizhong Wang
user
yizhongw
65bc696d7c6ad1994b7fb46e
Shane Arora
65bc696d7c6ad1994b7fb46f
David Atkinson
65bc696d7c6ad1994b7fb470
Russell Authur
admin_assigned
2024-02-02T09:30:52.208Z
62712c97bcef985363d9e368
/avatars/0fcc2f096bc6ba50a43cf1345d5c0066.svg
Russell Authur
user
rauthur
65bc696d7c6ad1994b7fb471
Khyathi Raghavi Chandu
admin_assigned
2024-02-02T09:31:01.103Z
62ebe6472f8999993a847bb6
/avatars/29fb590f8116bdc9c55f7a333e2c4280.svg
Khyathi Raghavi Chandu
user
khyathi
65bc696d7c6ad1994b7fb472
Arman Cohan
admin_assigned
2024-02-02T09:31:07.535Z
5f5ba21188f57f65f951f255
https://cdn-avatars.huggingface.co/v1/production/uploads/1599840760465-noauth.png
Arman Cohan
user
armanc
65bc696d7c6ad1994b7fb473
Jennifer Dumas
65bc696d7c6ad1994b7fb474
Yanai Elazar
admin_assigned
2024-02-02T09:31:23.561Z
623ca115a795593324c4353f
/avatars/bf11fe728df2786d52ed4d2de12b48d3.svg
Yanai Elazar
user
yanaiela
65bc696d7c6ad1994b7fb475
Yuling Gu
admin_assigned
2024-02-02T11:10:37.473Z
62a8b91c1277a49db0c5c080
/avatars/9cf834a02b67af862659c9b9d4367a52.svg
Yuling Gu
user
SaveBertAndGpt
65bc696d7c6ad1994b7fb476
Jack Hessel
admin_assigned
2024-02-02T11:10:46.037Z
625498644f4edf771516b2cb
https://cdn-avatars.huggingface.co/v1/production/uploads/1664497251391-625498644f4edf771516b2cb.jpeg
Jack Hessel
user
jmhessel
65bc696d7c6ad1994b7fb477
Tushar Khot
admin_assigned
2024-02-02T11:10:52.780Z
62cdf2dbaac2c91c95581830
https://cdn-avatars.huggingface.co/v1/production/uploads/62cdf2dbaac2c91c95581830/N3NuO7j53qkauJOf3kkRS.jpeg
Tushar Khot
user
tusharkhot
65bc696d7c6ad1994b7fb478
William Merrill
65bc696d7c6ad1994b7fb479
Jacob Morrison
admin_assigned
2024-02-02T11:11:13.650Z
63f51f6cb51da4d61da72937
/avatars/6023ecaafb28b7599f59816381d58bd0.svg
Jacob Morrison
user
jacobmorrison
65bc696d7c6ad1994b7fb47a
Niklas Muennighoff
admin_assigned
2024-02-02T11:11:23.013Z
5f1eb362eec0ad2a071ad6e2
https://cdn-avatars.huggingface.co/v1/production/uploads/5f1eb362eec0ad2a071ad6e2/IXMYkYKuTwn6kBdWnQeeY.png
Niklas Muennighoff
user
Muennighoff
65bc696d7c6ad1994b7fb47b
Aakanksha Naik
admin_assigned
2024-02-02T11:11:51.673Z
6268981926476157a9262a73
/avatars/a6e4d81580039e8680c194d89a716535.svg
Aakanksha Naik
user
aakanksha
65bc696d7c6ad1994b7fb47c
Crystal Nam
admin_assigned
2024-02-02T11:12:00.866Z
64a7264f44387a55046f92e5
/avatars/7f6e076a2a535957bc9c7ac4aeae716d.svg
Crystal Nam
user
CrystalN
65bc696d7c6ad1994b7fb47d
Matthew E. Peters
admin_assigned
2024-02-02T11:12:35.568Z
63d0455a8cf6c8e4d2087eeb
/avatars/e86cb0ffbfa3c64f162945c40a2fdb89.svg
Matthew Peters
user
matt-peters
65bc696d7c6ad1994b7fb47e
Valentina Pyatkin
admin_assigned
2024-02-02T11:12:48.140Z
6556cff80e7a7067a934445f
https://cdn-avatars.huggingface.co/v1/production/uploads/6556cff80e7a7067a934445f/PoT7qQ6tVqLGGrYvGBbr8.jpeg
Valentina Pyatkin
user
valpy
65bc696d7c6ad1994b7fb47f
Abhilasha Ravichander
admin_assigned
2024-02-02T11:12:55.122Z
6349886c429608888c42319a
/avatars/f84b5fe8b76172878274754e3399d6ec.svg
Abhilasha Ravichander
user
lasha-nlp
65bc696d7c6ad1994b7fb480
Dustin Schwenk
admin_assigned
2024-02-02T11:13:26.572Z
647e791b7f9ad5e44babb965
https://cdn-avatars.huggingface.co/v1/production/uploads/647e791b7f9ad5e44babb965/Vjfy4c3ooDwU24N7mvRdy.jpeg
Dustin Schwenk
user
drschwenk
65bc696d7c6ad1994b7fb481
Saurabh Shah
admin_assigned
2024-02-02T11:13:47.158Z
65244773e6e5f6b1033bb371
/avatars/2366893a2c6503c3894be5c427d9cbbe.svg
Saurabh Shah
user
saurabh-shah
65bc696d7c6ad1994b7fb482
Will Smith
65bc696d7c6ad1994b7fb483
Emma Strubell
65bc696d7c6ad1994b7fb484
Nishant Subramani
admin_assigned
2024-02-02T11:14:13.026Z
617b167b941993035fbaf2bf
https://cdn-avatars.huggingface.co/v1/production/uploads/617b167b941993035fbaf2bf/Wt3QMbjX9EMxdIrQdrF2l.jpeg
Nishant Subramani
user
nsubramani23
65bc696d7c6ad1994b7fb485
Mitchell Wortsman
65bc696d7c6ad1994b7fb486
Pradeep Dasigi
admin_assigned
2024-02-02T11:16:11.327Z
6408fcc93461c51cf735a61e
/avatars/619f3653911d111f046a5a6c30fc8319.svg
Pradeep Dasigi
user
pradeepd
65bc696d7c6ad1994b7fb487
Nathan Lambert
admin_assigned
2024-02-02T11:16:29.665Z
628e5f90a9a3c754c1f7c88f
https://cdn-avatars.huggingface.co/v1/production/uploads/628e5f90a9a3c754c1f7c88f/iWqMY_l6dalrgRaJZWbK3.png
Nathan Lambert
user
natolambert
65bc696d7c6ad1994b7fb488
Kyle Richardson
claimed_verified
2024-02-02T09:00:39.435Z
659c6f5e1d398a238152227d
/avatars/dba97fe8cb825102f1eae97104a71f64.svg
Kyle Richardson
user
yakazimir
65bc696d7c6ad1994b7fb489
Luke Zettlemoyer
65bc696d7c6ad1994b7fb48a
Jesse Dodge
admin_assigned
2024-02-02T11:16:38.638Z
6283f38567d336d3e5d5280e
/avatars/d0a54aaec74a90b050e671c191b87a80.svg
Jesse Dodge
user
JesseDodge
65bc696d7c6ad1994b7fb48b
Kyle Lo
65bc696d7c6ad1994b7fb48c
Luca Soldaini
admin_assigned
2024-02-02T11:16:51.964Z
5f04d8c45d08220171a0ad32
https://cdn-avatars.huggingface.co/v1/production/uploads/5f04d8c45d08220171a0ad32/uXEta6nqBabrUlAOXnS5g.jpeg
Luca Soldaini
user
soldni
65bc696d7c6ad1994b7fb48d
Noah A. Smith
65bc696d7c6ad1994b7fb48e
Hannaneh Hajishirzi
OLMo: Accelerating the Science of Language Models
Language models (LMs) have become ubiquitous in both NLP research and in
commercial product offerings. As their commercial importance has surged, the
most powerful models have become closed off, gated behind proprietary
interfaces, with important details of their training data, architectures, and
development undisclosed. Given the importance of these details in
scientifically studying these models, including their biases and potential
risks, we believe it is essential for the research community to have access to
powerful, truly open LMs. To this end, this technical report details the first
release of OLMo, a state-of-the-art, truly Open Language Model and its
framework to build and study the science of language modeling. Unlike most
prior efforts that have only released model weights and inference code, we
release OLMo and the whole framework, including training data and training and
evaluation code. We hope this release will empower and strengthen the open
research community and inspire a new wave of innovation.
OLMo, an open-sourced language model, provides comprehensive access to training data, code, and architecture, facilitating research and innovation in language modeling.
open language model language models training data training code evaluation code