title: StarCoder: may the source be with you!
summary: The BigCode community, an open-scientific collaboration working on the
  responsible development of Large Language Models for Code (Code LLMs),
  introduces StarCoder and StarCoderBase: 15.5B parameter models with 8K context
  length, infilling capabilities and fast large-batch inference enabled by
  multi-query attention. StarCoderBase is trained on 1 trillion tokens sourced
  from The Stack, a large collection of permissively licensed GitHub repositories
  with inspection tools and an opt-out process. We fine-tuned StarCoderBase on
  35B Python tokens, resulting in the creation of StarCoder. We perform the most
  comprehensive evaluation of Code LLMs to date and show that StarCoderBase
  outperforms every open Code LLM that supports multiple programming languages
  and matches or outperforms the OpenAI code-cushman-001 model. Furthermore,
  StarCoder outperforms every model that is fine-tuned on Python, can be prompted
  to achieve 40\% pass@1 on HumanEval, and still retains its performance on other
  programming languages. We take several important steps towards a safe
  open-access model release, including an improved PII redaction pipeline and a
  novel attribution tracing tool, and make the StarCoder models publicly
  available under a more commercially viable version of the Open Responsible AI
  Model license.
paper_authors._id: 645c4c127848314a460d1759
paper_authors.name: Raymond Li
paper_authors._id: 645c4c127848314a460d175a
paper_authors.name: Loubna Ben Allal
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-11T08:27:56.619Z
paper_authors._id: 61c141342aac764ce1654e43
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61c141342aac764ce1654e43/81AwoT5IQ_Xdw0OVw7TKu.jpeg
paper_authors.fullname: Loubna Ben Allal
paper_authors.type: user
paper_authors.user: loubnabnl
paper_authors._id: 645c4c127848314a460d175b
paper_authors.name: Yangtian Zi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-15T07:34:40.656Z
paper_authors._id: 6354a759eca5a823e3bc89ba
paper_authors.avatarUrl: /avatars/e4ad8a56bf416ab11a1cfaf64f2edc09.svg
paper_authors.fullname: Yangtian Zi
paper_authors.type: user
paper_authors.user: ytzi
paper_authors._id: 645c4c127848314a460d175c
paper_authors.name: Niklas Muennighoff
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-11T08:29:15.774Z
paper_authors._id: 5f1eb362eec0ad2a071ad6e2
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/5f1eb362eec0ad2a071ad6e2/IXMYkYKuTwn6kBdWnQeeY.png
paper_authors.fullname: Niklas Muennighoff
paper_authors.type: user
paper_authors.user: Muennighoff
paper_authors._id: 645c4c127848314a460d175d
paper_authors.name: Denis Kocetkov
paper_authors._id: 645c4c127848314a460d175e
paper_authors.name: Chenghao Mou
paper_authors._id: 645c4c127848314a460d175f
paper_authors.name: Marc Marone
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-11T15:00:49.640Z
paper_authors._id: 63e410e88083f19a218be964
paper_authors.avatarUrl: /avatars/ce271d1686f6e5ee1f5b2d429cb60de6.svg
paper_authors.fullname: Marc Marone
paper_authors.type: user
paper_authors.user: mmarone
paper_authors._id: 645c4c127848314a460d1760
paper_authors.name: Christopher Akiki
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-11T08:28:12.968Z
paper_authors._id: 5e70f6048ce3c604d78fe133
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1646492542174-5e70f6048ce3c604d78fe133.jpeg
paper_authors.fullname: Christopher Akiki
paper_authors.type: user
paper_authors.user: christopher
paper_authors._id: 645c4c127848314a460d1761
paper_authors.name: Jia Li
paper_authors._id: 645c4c127848314a460d1762
paper_authors.name: Jenny Chim
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-15T15:28:34.958Z
paper_authors._id: 5fc8e6e9628d967e353416b3
paper_authors.avatarUrl: /avatars/d913dff9380e711aa4ecf368e3070f88.svg
paper_authors.fullname: Jenny Chim
paper_authors.type: user
paper_authors.user: j-chim
paper_authors._id: 645c4c127848314a460d1763
paper_authors.name: Qian Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-11T08:25:20.910Z
paper_authors._id: 612ee6a7b960e78c6d2319d4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/612ee6a7b960e78c6d2319d4/2Hu9BaAyXbyh1vt0v1Qui.jpeg
paper_authors.fullname: Qian Liu
paper_authors.type: user
paper_authors.user: SivilTaram
paper_authors._id: 645c4c127848314a460d1764
paper_authors.name: Evgenii Zheltonozhskii
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-11T15:00:36.990Z
paper_authors._id: 61ae250602b8655e3c8973cd
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61ae250602b8655e3c8973cd/aXp22-B-oKlm_h6kLx9Uv.jpeg
paper_authors.fullname: Evgenii Zheltonozhskii
paper_authors.type: user
paper_authors.user: EvgeniyZh
paper_authors._id: 645c4c127848314a460d1765
paper_authors.name: Terry Yue Zhuo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-15T07:34:57.914Z
paper_authors._id: 62b7fb545233925f253531c8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62b7fb545233925f253531c8/W50u2G1HK3EtUKHRU189V.jpeg
paper_authors.fullname: Terry Yue Zhuo
paper_authors.type: user
paper_authors.user: terryyz
paper_authors._id: 645c4c127848314a460d1766
paper_authors.name: Thomas Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-11T08:28:52.145Z
paper_authors._id: 61140e35d08630d2676c9829
paper_authors.avatarUrl: /avatars/3d6e4b4d02eda7b5ef28e1cc0fb8e08a.svg
paper_authors.fullname: Thomas Wang
paper_authors.type: user
paper_authors.user: TimeRobber
paper_authors._id: 645c4c127848314a460d1767
paper_authors.name: Olivier Dehaene
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-11T08:28:32.574Z
paper_authors._id: 62a093d63e7d1dda047039fc
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62a093d63e7d1dda047039fc/QGpVSKuJLwl2EsiffCYML.jpeg
paper_authors.fullname: Olivier Dehaene
paper_authors.type: user
paper_authors.user: olivierdehaene
paper_authors._id: 645c4c127848314a460d1768
paper_authors.name: Mishig Davaadorj
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-11T08:34:42.462Z
paper_authors._id: 60a551a34ecc5d054c8ad93e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60a551a34ecc5d054c8ad93e/dhcBFtwNLcKqqASxniyVw.jpeg
paper_authors.fullname: Mishig Davaadorj
paper_authors.type: user
paper_authors.user: mishig
paper_authors._id: 645c4c127848314a460d1769
paper_authors.name: Joel Lamy-Poirier
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-12T08:37:39.601Z
paper_authors._id: 6372ec0e759c1ca82045c43d
paper_authors.avatarUrl: /avatars/86d5dbdb10a0cbf9a39a32e2d87361b6.svg
paper_authors.fullname: Joel Lamy-Poirier
paper_authors.type: user
paper_authors.user: jlamypoirier
paper_authors._id: 645c4c127848314a460d176a
paper_authors.name: João Monteiro
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-11T15:00:27.548Z
paper_authors._id: 63347fed495073b88704cf02
paper_authors.avatarUrl: /avatars/cb5545adaf1cfe0166119c586f4ac9c8.svg
paper_authors.fullname: João Monteiro
paper_authors.type: user
paper_authors.user: joaomonteirof
paper_authors._id: 645c4c127848314a460d176b
paper_authors.name: Oleh Shliazhko
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-11T15:00:46.540Z
paper_authors._id: 630e7fb01ef92d4e37a56400
paper_authors.avatarUrl: /avatars/418a19c9a0deaba6ee4938e5c941253e.svg
paper_authors.fullname: Oleh Shliazhko
paper_authors.type: user
paper_authors.user: olmer
paper_authors._id: 645c4c127848314a460d176c
paper_authors.name: Nicolas Gontier
paper_authors._id: 645c4c127848314a460d176d
paper_authors.name: Nicholas Meade
paper_authors._id: 645c4c127848314a460d176e
paper_authors.name: Armel Zebaze
paper_authors._id: 645c4c127848314a460d176f
paper_authors.name: Ming-Ho Yee
paper_authors._id: 645c4c127848314a460d1770
paper_authors.name: Logesh Kumar Umapathi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-11T15:00:52.569Z
paper_authors._id: 6052dd84a7226b25aaeea2be
paper_authors.avatarUrl: /avatars/d130e7051d81303130883709aacb2618.svg
paper_authors.fullname: Logesh Kumar umapathi
paper_authors.type: user
paper_authors.user: infinitylogesh
paper_authors._id: 645c4c127848314a460d1771
paper_authors.name: Jian Zhu
paper_authors._id: 645c4c127848314a460d1772
paper_authors.name: Benjamin Lipkin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-11T15:00:56.047Z
paper_authors._id: 613a6efea488863c1920731e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1664283416127-613a6efea488863c1920731e.jpeg
paper_authors.fullname: Ben Lipkin
paper_authors.type: user
paper_authors.user: benlipkin
paper_authors._id: 645c4c127848314a460d1773
paper_authors.name: Muhtasham Oblokulov
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-11T15:00:00.456Z
paper_authors._id: 5fcaabed246881afd5b00167
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1651847561574-5fcaabed246881afd5b00167.jpeg
paper_authors.fullname: Muhtasham Oblokulov
paper_authors.type: user
paper_authors.user: muhtasham
paper_authors._id: 645c4c127848314a460d1774
paper_authors.name: Zhiruo Wang
paper_authors._id: 645c4c127848314a460d1775
paper_authors.name: Rudra Murthy
paper_authors._id: 645c4c127848314a460d1776
paper_authors.name: Jason Stillerman
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-15T07:34:52.595Z
paper_authors._id: 6178d842267320abb99df776
paper_authors.avatarUrl: /avatars/8739705ef7eabc945a1ef3c07196af13.svg
paper_authors.fullname: Jason Stillerman
paper_authors.type: user
paper_authors.user: stillerman
paper_authors._id: 645c4c127848314a460d1777
paper_authors.name: Siva Sankalp Patel
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-12T08:11:30.592Z
paper_authors._id: 618c2037526de8dfed6b3180
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/618c2037526de8dfed6b3180/QhobNWuHwVzyG9OtntzSp.jpeg
paper_authors.fullname: Siva Sankalp
paper_authors.type: user
paper_authors.user: sivasankalpp
paper_authors._id: 645c4c127848314a460d1778
paper_authors.name: Dmitry Abulkhanov
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-19T10:28:18.648Z
paper_authors._id: 636a9a07e3ad78bc68b1a5a2
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1668020490988-636a9a07e3ad78bc68b1a5a2.jpeg
paper_authors.fullname: Dmitry Abulkhanov
paper_authors.type: user
paper_authors.user: mponty
paper_authors._id: 645c4c127848314a460d1779
paper_authors.name: Marco Zocca
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-30T20:59:21.536Z
paper_authors._id: 6282a048e84c35bcb7d90a76
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1665103679595-6282a048e84c35bcb7d90a76.jpeg
paper_authors.fullname: Marco Zocca
paper_authors.type: user
paper_authors.user: ocramz
paper_authors._id: 645c4c127848314a460d177a
paper_authors.name: Manan Dey
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-11T15:00:24.018Z
paper_authors._id: 5fc53305fa6eef7667a4d691
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1623089209793-5fc53305fa6eef7667a4d691.jpeg
paper_authors.fullname: Manan Dey
paper_authors.type: user
paper_authors.user: manandey
paper_authors._id: 645c4c127848314a460d177b
paper_authors.name: Zhihan Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-12T08:11:22.127Z
paper_authors._id: 63bf9695da08ed054400205e
paper_authors.avatarUrl: /avatars/b6fca49559a61cf66628088c60d26c10.svg
paper_authors.fullname: Zhihan Zhang
paper_authors.type: user
paper_authors.user: zhihz0535
paper_authors._id: 645c4c127848314a460d177c
paper_authors.name: Nour Fahmy
paper_authors._id: 645c4c127848314a460d177d
paper_authors.name: Urvashi Bhattacharyya
paper_authors._id: 645c4c127848314a460d177e
paper_authors.name: Wenhao Yu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-11T19:01:44.456Z
paper_authors._id: 5feab3a28a3201f8e554c969
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1660795228685-5feab3a28a3201f8e554c969.png
paper_authors.fullname: Wenhao Yu
paper_authors.type: user
paper_authors.user: wyu1
paper_authors._id: 645c4c127848314a460d177f
paper_authors.name: Swayam Singh
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-11T08:25:18.334Z
paper_authors._id: 62f47c093561a52aa5a67c90
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62f47c093561a52aa5a67c90/0JbnPzHI345sqnrmrxmCw.jpeg
paper_authors.fullname: Swayam Singh
paper_authors.type: user
paper_authors.user: rootacess
paper_authors._id: 645c4c127848314a460d1780
paper_authors.name: Sasha Luccioni
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-11T08:34:28.553Z
paper_authors._id: 60edd0133e2c73a9a21455f5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60edd0133e2c73a9a21455f5/yK1G-Fv-YjYb7v_chkz3p.jpeg
paper_authors.fullname: Sasha Luccioni
paper_authors.type: user
paper_authors.user: sasha
paper_authors._id: 645c4c127848314a460d1781
paper_authors.name: Paulo Villegas
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-15T07:34:55.173Z
paper_authors._id: 60d433541af6803389820846
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1626374472249-60d433541af6803389820846.jpeg
paper_authors.fullname: Paulo Villegas
paper_authors.type: user
paper_authors.user: Paulo
paper_authors._id: 645c4c127848314a460d1782
paper_authors.name: Maxim Kunakov
paper_authors._id: 645c4c127848314a460d1783
paper_authors.name: Fedor Zhdanov
paper_authors._id: 645c4c127848314a460d1784
paper_authors.name: Manuel Romero
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-11T08:30:10.779Z
paper_authors._id: 5e4318d616b09a31220980d6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/5e4318d616b09a31220980d6/24rMJ_vPh3gW9ZEmj64xr.png
paper_authors.fullname: Manuel Romero
paper_authors.type: user
paper_authors.user: mrm8488
paper_authors._id: 645c4c127848314a460d1785
paper_authors.name: Tony Lee
paper_authors._id: 645c4c127848314a460d1786
paper_authors.name: Nadav Timor
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-12T08:11:16.640Z
paper_authors._id: 62be5fc0c990a0a515da5f34
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1674221847840-62be5fc0c990a0a515da5f34.jpeg
paper_authors.fullname: Nadav Timor
paper_authors.type: user
paper_authors.user: Nadav-Timor
paper_authors._id: 645c4c127848314a460d1787
paper_authors.name: Jennifer Ding
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-06T09:05:59.294Z
paper_authors._id: 62a0da842e30aaf94ebaaa12
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1679073278459-62a0da842e30aaf94ebaaa12.jpeg
paper_authors.fullname: Jennifer Ding
paper_authors.type: user
paper_authors.user: jending12
paper_authors._id: 645c4c127848314a460d1788
paper_authors.name: Claire Schlesinger
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-15T07:34:49.659Z
paper_authors._id: 63d4aa23ff1384ce6c683544
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63d4aa23ff1384ce6c683544/7RQIh5dyak8wanCdS1dGu.jpeg
paper_authors.fullname: Claire Schlesinger
paper_authors.type: user
paper_authors.user: esslushy
paper_authors._id: 645c4c127848314a460d1789
paper_authors.name: Hailey Schoelkopf
paper_authors._id: 645c4c127848314a460d178a
paper_authors.name: Jan Ebert
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-11T15:00:40.385Z
paper_authors._id: 630dc2eda16f7b175da838dd
paper_authors.avatarUrl: /avatars/11f515923b3d378c056d99ff7e52d6aa.svg
paper_authors.fullname: Jan
paper_authors.type: user
paper_authors.user: jaeb
paper_authors._id: 645c4c127848314a460d178b
paper_authors.name: Tri Dao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-14T15:33:40.468Z
paper_authors._id: 64b8a6b5cf14c2fabe98159b
paper_authors.avatarUrl: /avatars/dbc009451865435bf290791beadc4723.svg
paper_authors.fullname: Tri Dao
paper_authors.type: user
paper_authors.user: tridao
paper_authors._id: 645c4c127848314a460d178c
paper_authors.name: Mayank Mishra
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-11T15:00:34.183Z
paper_authors._id: 62cd5057674cdb524450093d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62cd5057674cdb524450093d/f67rlrdsKPRTLdXCXoa_X.jpeg
paper_authors.fullname: Mayank Mishra
paper_authors.type: user
paper_authors.user: mayank-mishra
paper_authors._id: 645c4c127848314a460d178d
paper_authors.name: Alex Gu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-19T08:54:48.209Z
paper_authors._id: 6179abf8ec6ce4dc2e5f2376
paper_authors.avatarUrl: /avatars/d10c6a1b350146b36949a24220471295.svg
paper_authors.fullname: Alex Gu
paper_authors.type: user
paper_authors.user: minimario
paper_authors._id: 645c4c127848314a460d178e
paper_authors.name: Jennifer Robinson
paper_authors._id: 645c4c127848314a460d178f
paper_authors.name: Carolyn Jane Anderson
paper_authors._id: 645c4c127848314a460d1790
paper_authors.name: Brendan Dolan-Gavitt
paper_authors._id: 645c4c127848314a460d1791
paper_authors.name: Danish Contractor
paper_authors._id: 645c4c127848314a460d1792
paper_authors.name: Siva Reddy
paper_authors._id: 645c4c127848314a460d1793
paper_authors.name: Daniel Fried
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-11T19:01:47.280Z
paper_authors._id: 623e041cdcda6a7153052d02
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/623e041cdcda6a7153052d02/dCAe49oOZDsyp40xqiB-L.png
paper_authors.fullname: Daniel Fried
paper_authors.type: user
paper_authors.user: dpfried
paper_authors._id: 645c4c127848314a460d1794
paper_authors.name: Dzmitry Bahdanau
paper_authors._id: 645c4c127848314a460d1795
paper_authors.name: Yacine Jernite
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-11T08:33:52.890Z
paper_authors._id: 5ee3a7cd2a3eae3cbdad1305
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1594144055859-5ee3a7cd2a3eae3cbdad1305.jpeg
paper_authors.fullname: Yacine Jernite
paper_authors.type: user
paper_authors.user: yjernite
paper_authors._id: 645c4c127848314a460d1796
paper_authors.name: Carlos Muñoz Ferrandis
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-11T08:29:34.163Z
paper_authors._id: 629f05bddaca51e26a7cfef1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1654588848873-noauth.jpeg
paper_authors.fullname: Carlos Muñoz Ferrandis
paper_authors.type: user
paper_authors.user: CarlosMF
paper_authors._id: 645c4c127848314a460d1797
paper_authors.name: Sean Hughes
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-11T19:01:36.853Z
paper_authors._id: 633dd1e467e67df066ef2cbf
paper_authors.avatarUrl: /avatars/121fff6aefd09565fd2b01d7fcda6757.svg
paper_authors.fullname: Sean Hughes
paper_authors.type: user
paper_authors.user: hughesthe1st
paper_authors._id: 645c4c127848314a460d1798
paper_authors.name: Thomas Wolf
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-11T08:35:06.520Z
paper_authors._id: 5df7e9e5da6d0311fd3d53f9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1583857746553-5df7e9e5da6d0311fd3d53f9.jpeg
paper_authors.fullname: Thomas Wolf
paper_authors.type: user
paper_authors.user: thomwolf
paper_authors._id: 645c4c127848314a460d1799
paper_authors.name: Arjun Guha
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-11T15:00:31.316Z
paper_authors._id: 62d8315bad693a1a962864b3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1664332914111-62d8315bad693a1a962864b3.png
paper_authors.fullname: Arjun Guha
paper_authors.type: user
paper_authors.user: arjunguha
paper_authors._id: 645c4c127848314a460d179a
paper_authors.name: Leandro von Werra
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-11T08:25:14.980Z
paper_authors._id: 5e48005437cb5b49818287a5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/5e48005437cb5b49818287a5/4uCXGGui-9QifAT4qelxU.png
paper_authors.fullname: Leandro von Werra
paper_authors.type: user
paper_authors.user: lvwerra
paper_authors._id: 645c4c127848314a460d179b
paper_authors.name: Harm de Vries
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-12T08:37:03.497Z
paper_authors._id: 62ce91e8b5811ba53c0ce7b9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62ce91e8b5811ba53c0ce7b9/ZNgd-p0h-q1j374QqEGrb.jpeg
paper_authors.fullname: Harm de Vries
paper_authors.type: user
paper_authors.user: harmdevries
paper_title: StarCoder: may the source be with you!
paper_summary: The BigCode community, an open-scientific collaboration working on the
  responsible development of Large Language Models for Code (Code LLMs),
  introduces StarCoder and StarCoderBase: 15.5B parameter models with 8K context
  length, infilling capabilities and fast large-batch inference enabled by
  multi-query attention. StarCoderBase is trained on 1 trillion tokens sourced
  from The Stack, a large collection of permissively licensed GitHub repositories
  with inspection tools and an opt-out process. We fine-tuned StarCoderBase on
  35B Python tokens, resulting in the creation of StarCoder. We perform the most
  comprehensive evaluation of Code LLMs to date and show that StarCoderBase
  outperforms every open Code LLM that supports multiple programming languages
  and matches or outperforms the OpenAI code-cushman-001 model. Furthermore,
  StarCoder outperforms every model that is fine-tuned on Python, can be prompted
  to achieve 40\% pass@1 on HumanEval, and still retains its performance on other
  programming languages. We take several important steps towards a safe
  open-access model release, including an improved PII redaction pipeline and a
  novel attribution tracing tool, and make the StarCoder models publicly
  available under a more commercially viable version of the Open Responsible AI
  Model license.
paper_ai_keywords: StarCoder StarCoderBase parameter models context length infilling capabilities multi-query attention The Stack GitHub repositories fine-tuned evaluation Code LLMs HumanEval PII redaction pipeline attribution tracing tool Open Responsible AI Model license