title: DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code
    Intelligence
summary: We present DeepSeek-Coder-V2, an open-source Mixture-of-Experts (MoE) code
  language model that achieves performance comparable to GPT4-Turbo in
  code-specific tasks. Specifically, DeepSeek-Coder-V2 is further pre-trained
  from an intermediate checkpoint of DeepSeek-V2 with additional 6 trillion
  tokens. Through this continued pre-training, DeepSeek-Coder-V2 substantially
  enhances the coding and mathematical reasoning capabilities of DeepSeek-V2,
  while maintaining comparable performance in general language tasks. Compared to
  DeepSeek-Coder-33B, DeepSeek-Coder-V2 demonstrates significant advancements in
  various aspects of code-related tasks, as well as reasoning and general
  capabilities. Additionally, DeepSeek-Coder-V2 expands its support for
  programming languages from 86 to 338, while extending the context length from
  16K to 128K. In standard benchmark evaluations, DeepSeek-Coder-V2 achieves
  superior performance compared to closed-source models such as GPT4-Turbo,
  Claude 3 Opus, and Gemini 1.5 Pro in coding and math benchmarks.
paper_authors._id: 66724bb98b5e22ce4f66df3e
paper_authors.name: DeepSeek-AI
paper_authors._id: 66724bb98b5e22ce4f66df3f
paper_authors.name: Qihao Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-19T08:26:08.420Z
paper_authors._id: 63cd76b4374057a338e8e703
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63cd76b4374057a338e8e703/i4Qk5-0aYx3oRhC8b50aJ.jpeg
paper_authors.fullname: zhuqihao
paper_authors.type: user
paper_authors.user: zqh11
paper_authors._id: 66724bb98b5e22ce4f66df40
paper_authors.name: Daya Guo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-19T08:26:15.136Z
paper_authors._id: 653df20eaa1f487614da4db1
paper_authors.avatarUrl: /avatars/12b27ce2c59f53b7e464039deab36a5d.svg
paper_authors.fullname: Daya Guo
paper_authors.type: user
paper_authors.user: guoday
paper_authors._id: 66724bb98b5e22ce4f66df41
paper_authors.name: Zhihong Shao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-19T08:26:29.442Z
paper_authors._id: 65db64f8b62d242ed8711701
paper_authors.avatarUrl: /avatars/753e9f980eb6786c6b53b2f1becbf745.svg
paper_authors.fullname: Zhihong Shao
paper_authors.type: user
paper_authors.user: ZhihongShao
paper_authors._id: 66724bb98b5e22ce4f66df42
paper_authors.name: Dejian Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-19T08:26:41.738Z
paper_authors._id: 6225bb44c6e650de3a65dbaa
paper_authors.avatarUrl: /avatars/99c99ced2461978df572c27c1b3a4904.svg
paper_authors.fullname: DejianYang
paper_authors.type: user
paper_authors.user: DejianYang
paper_authors._id: 66724bb98b5e22ce4f66df43
paper_authors.name: Peiyi Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-19T08:26:48.355Z
paper_authors._id: 656873f33fd0bf1f82558695
paper_authors.avatarUrl: /avatars/7a085da2e2a91d7f41988501a573ebf9.svg
paper_authors.fullname: PEIYI, WANG
paper_authors.type: user
paper_authors.user: peiyiwang89
paper_authors._id: 66724bb98b5e22ce4f66df44
paper_authors.name: Runxin Xu
paper_authors._id: 66724bb98b5e22ce4f66df45
paper_authors.name: Y. Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-19T20:20:49.778Z
paper_authors._id: 640556390ab5e22719fc65c8
paper_authors.avatarUrl: /avatars/c1814c42ee5fdcc926f841881936f618.svg
paper_authors.fullname: Yu Wu
paper_authors.type: user
paper_authors.user: wumark
paper_authors._id: 66724bb98b5e22ce4f66df46
paper_authors.name: Yukun Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-19T08:27:08.329Z
paper_authors._id: 664c65145312449198fb79a8
paper_authors.avatarUrl: /avatars/b2524700c4fe02c8fbcabdd0b4a0f587.svg
paper_authors.fullname: yukun li
paper_authors.type: user
paper_authors.user: imli
paper_authors._id: 66724bb98b5e22ce4f66df47
paper_authors.name: Huazuo Gao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-19T08:27:14.990Z
paper_authors._id: 64e370be59aa5366642ac329
paper_authors.avatarUrl: /avatars/0fa1eb6ac6c1aeff3e65bc86a6617f64.svg
paper_authors.fullname: Huazuo Gao
paper_authors.type: user
paper_authors.user: gaohuazuo
paper_authors._id: 66724bb98b5e22ce4f66df48
paper_authors.name: Shirong Ma
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-19T08:27:32.842Z
paper_authors._id: 6482e57a04f67f5f6056a61b
paper_authors.avatarUrl: /avatars/b26faf19ba1493b91102ac7978ab3230.svg
paper_authors.fullname: Shirong Ma
paper_authors.type: user
paper_authors.user: msr2000
paper_authors._id: 66724bb98b5e22ce4f66df49
paper_authors.name: Wangding Zeng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-19T08:27:39.188Z
paper_authors._id: 659384dec0b1372b2e3c184d
paper_authors.avatarUrl: /avatars/fdbff8012fe80d5f2e861cbfa2675125.svg
paper_authors.fullname: wangding zeng
paper_authors.type: user
paper_authors.user: zwd973-deepseek
paper_authors._id: 66724bb98b5e22ce4f66df4a
paper_authors.name: Xiao Bi
paper_authors._id: 66724bb98b5e22ce4f66df4b
paper_authors.name: Zihui Gu
paper_authors._id: 66724bb98b5e22ce4f66df4c
paper_authors.name: Hanwei Xu
paper_authors._id: 66724bb98b5e22ce4f66df4d
paper_authors.name: Damai Dai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-19T08:28:06.857Z
paper_authors._id: 659389f8de82e1ef7b9a8b13
paper_authors.avatarUrl: /avatars/896ed9f4cdbd317493b303d070b7e12a.svg
paper_authors.fullname: Damai Dai
paper_authors.type: user
paper_authors.user: DeepSeekDDM
paper_authors._id: 66724bb98b5e22ce4f66df4e
paper_authors.name: Kai Dong
paper_authors._id: 66724bb98b5e22ce4f66df4f
paper_authors.name: Liyue Zhang
paper_authors._id: 66724bb98b5e22ce4f66df50
paper_authors.name: Yishi Piao
paper_authors._id: 66724bb98b5e22ce4f66df51
paper_authors.name: Zhibin Gou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-19T07:38:03.852Z
paper_authors._id: 62dcf5d4169bd1d2ef2ca724
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62dcf5d4169bd1d2ef2ca724/oRFFmJDJTLYtPRVPCweQ_.jpeg
paper_authors.fullname: Zhibin Gou
paper_authors.type: user
paper_authors.user: zubingou
paper_authors._id: 66724bb98b5e22ce4f66df52
paper_authors.name: Zhenda Xie
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-19T08:28:37.958Z
paper_authors._id: 6539f6ea26df26ecd1393c37
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6539f6ea26df26ecd1393c37/9VJusLLAiLhUxfBAFggpF.jpeg
paper_authors.fullname: Zhenda Xie
paper_authors.type: user
paper_authors.user: zdaxie
paper_authors._id: 66724bb98b5e22ce4f66df53
paper_authors.name: Zhewen Hao
paper_authors._id: 66724bb98b5e22ce4f66df54
paper_authors.name: Bingxuan Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-19T08:28:51.550Z
paper_authors._id: 6523d81d56fe05f216a559f6
paper_authors.avatarUrl: /avatars/07fcf56b5b8a0b64c31bdfe8fbf41cc6.svg
paper_authors.fullname: Bingxuan Wang
paper_authors.type: user
paper_authors.user: YellowDoge
paper_authors._id: 66724bb98b5e22ce4f66df55
paper_authors.name: Junxiao Song
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-19T08:28:58.526Z
paper_authors._id: 6565a2dd131d13ccc5d8cb12
paper_authors.avatarUrl: /avatars/f5c5441ba74791b64c9740911f952bac.svg
paper_authors.fullname: Junxiao Song
paper_authors.type: user
paper_authors.user: haha-point
paper_authors._id: 66724bb98b5e22ce4f66df56
paper_authors.name: Deli Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-19T08:29:05.063Z
paper_authors._id: 64ec0b6307987950ae738083
paper_authors.avatarUrl: /avatars/e131cd22b55e4b62cc48779e512c3da1.svg
paper_authors.fullname: deli chen
paper_authors.type: user
paper_authors.user: deli96
paper_authors._id: 66724bb98b5e22ce4f66df57
paper_authors.name: Xin Xie
paper_authors._id: 66724bb98b5e22ce4f66df58
paper_authors.name: Kang Guan
paper_authors._id: 66724bb98b5e22ce4f66df59
paper_authors.name: Yuxiang You
paper_authors._id: 66724bb98b5e22ce4f66df5a
paper_authors.name: Aixin Liu
paper_authors._id: 66724bb98b5e22ce4f66df5b
paper_authors.name: Qiushi Du
paper_authors._id: 66724bb98b5e22ce4f66df5c
paper_authors.name: Wenjun Gao
paper_authors._id: 66724bb98b5e22ce4f66df5d
paper_authors.name: Xuan Lu
paper_authors._id: 66724bb98b5e22ce4f66df5e
paper_authors.name: Qinyu Chen
paper_authors._id: 66724bb98b5e22ce4f66df5f
paper_authors.name: Yaohui Wang
paper_authors._id: 66724bb98b5e22ce4f66df60
paper_authors.name: Chengqi Deng
paper_authors._id: 66724bb98b5e22ce4f66df61
paper_authors.name: Jiashi Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-19T08:29:13.424Z
paper_authors._id: 64fca5f28d50404bc42ca78a
paper_authors.avatarUrl: /avatars/ae01ac0296d6ce1277dacb6894f570b8.svg
paper_authors.fullname: Jiashi Li
paper_authors.type: user
paper_authors.user: Beginlner
paper_authors._id: 66724bb98b5e22ce4f66df62
paper_authors.name: Chenggang Zhao
paper_authors._id: 66724bb98b5e22ce4f66df63
paper_authors.name: Chong Ruan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-17T05:38:42.320Z
paper_authors._id: 6398203609f12714ed1935c2
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6398203609f12714ed1935c2/uXgl0LgKnFYjq1Wz39-a6.jpeg
paper_authors.fullname: Chong Ruan
paper_authors.type: user
paper_authors.user: Chester111
paper_authors._id: 66724bb98b5e22ce4f66df64
paper_authors.name: Fuli Luo
paper_authors._id: 66724bb98b5e22ce4f66df65
paper_authors.name: Wenfeng Liang
paper_title: DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code
    Intelligence
paper_summary: We present DeepSeek-Coder-V2, an open-source Mixture-of-Experts (MoE) code
  language model that achieves performance comparable to GPT4-Turbo in
  code-specific tasks. Specifically, DeepSeek-Coder-V2 is further pre-trained
  from an intermediate checkpoint of DeepSeek-V2 with additional 6 trillion
  tokens. Through this continued pre-training, DeepSeek-Coder-V2 substantially
  enhances the coding and mathematical reasoning capabilities of DeepSeek-V2,
  while maintaining comparable performance in general language tasks. Compared to
  DeepSeek-Coder-33B, DeepSeek-Coder-V2 demonstrates significant advancements in
  various aspects of code-related tasks, as well as reasoning and general
  capabilities. Additionally, DeepSeek-Coder-V2 expands its support for
  programming languages from 86 to 338, while extending the context length from
  16K to 128K. In standard benchmark evaluations, DeepSeek-Coder-V2 achieves
  superior performance compared to closed-source models such as GPT4-Turbo,
  Claude 3 Opus, and Gemini 1.5 Pro in coding and math benchmarks.
paper_ai_keywords: Mixture-of-Experts (MoE) code-specific tasks pre-trained coding capabilities mathematical reasoning capabilities general language tasks context length