title: Qwen2 Technical Report
summary: This report introduces the Qwen2 series, the latest addition to our large
  language models and large multimodal models. We release a comprehensive suite
  of foundational and instruction-tuned language models, encompassing a parameter
  range from 0.5 to 72 billion, featuring dense models and a Mixture-of-Experts
  model. Qwen2 surpasses most prior open-weight models, including its predecessor
  Qwen1.5, and exhibits competitive performance relative to proprietary models
  across diverse benchmarks on language understanding, generation, multilingual
  proficiency, coding, mathematics, and reasoning.
    The flagship model, Qwen2-72B, showcases remarkable performance: 84.2 on
  MMLU, 37.9 on GPQA, 64.6 on HumanEval, 89.5 on GSM8K, and 82.4 on BBH as a base
  language model. The instruction-tuned variant, Qwen2-72B-Instruct, attains 9.1
  on MT-Bench, 48.1 on Arena-Hard, and 35.7 on LiveCodeBench. Moreover, Qwen2
  demonstrates robust multilingual capabilities, proficient in approximately 30
  languages, spanning English, Chinese, Spanish, French, German, Arabic, Russian,
  Korean, Japanese, Thai, Vietnamese, and more, underscoring its versatility and
  global reach.
    To foster community innovation and accessibility, we have made the Qwen2
  model weights openly available on Hugging Face1 and ModelScope2, and the
  supplementary materials including example code on GitHub3. These platforms also
  include resources for quantization, fine-tuning, and deployment, facilitating a
  wide range of applications and research endeavors.
paper_authors._id: 6695e0de321386ed51decbda
paper_authors.name: An Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-23T16:29:54.776Z
paper_authors._id: 62088594a5943c8a8fc94560
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1644733028938-62088594a5943c8a8fc94560.png
paper_authors.fullname: An Yang
paper_authors.type: user
paper_authors.user: yangapku
paper_authors._id: 6695e0de321386ed51decbdb
paper_authors.name: Baosong Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:12:42.572Z
paper_authors._id: 64b0a77df12b47366663884c
paper_authors.avatarUrl: /avatars/a212ea862abb5966060e439dd0e7656f.svg
paper_authors.fullname: Baosong Yang
paper_authors.type: user
paper_authors.user: Baosong
paper_authors._id: 6695e0de321386ed51decbdc
paper_authors.name: Binyuan Hui
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-16T20:16:02.548Z
paper_authors._id: 61e4c4ca1ab24785ac11ba69
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61e4c4ca1ab24785ac11ba69/1Q1zhhyGSJ9RJG9MzwxVv.jpeg
paper_authors.fullname: Binyuan Hui
paper_authors.type: user
paper_authors.user: huybery
paper_authors._id: 6695e0de321386ed51decbdd
paper_authors.name: Bo Zheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:12:57.275Z
paper_authors._id: 62c695ad5aae1c624ca992e2
paper_authors.avatarUrl: /avatars/20d10fb3338e4bd4dc59e88a18cb2617.svg
paper_authors.fullname: Bo Zheng
paper_authors.type: user
paper_authors.user: bzheng
paper_authors._id: 6695e0de321386ed51decbde
paper_authors.name: Bowen Yu
paper_authors._id: 6695e0de321386ed51decbdf
paper_authors.name: Chang Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:13:44.162Z
paper_authors._id: 622892774323cef93a956a4a
paper_authors.avatarUrl: /avatars/e57ef5c3b0c4289988ccd42f14e54336.svg
paper_authors.fullname: chang zhou
paper_authors.type: user
paper_authors.user: jiemizc
paper_authors._id: 6695e0de321386ed51decbe0
paper_authors.name: Chengpeng Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:13:51.135Z
paper_authors._id: 65294b334d7cf551ac50d6a6
paper_authors.avatarUrl: /avatars/75d21e20b711b871616ef3850bb900b7.svg
paper_authors.fullname: ChengpengLi
paper_authors.type: user
paper_authors.user: ChengpengLi
paper_authors._id: 6695e0de321386ed51decbe1
paper_authors.name: Chengyuan Li
paper_authors._id: 6695e0de321386ed51decbe2
paper_authors.name: Dayiheng Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:14:16.139Z
paper_authors._id: 6434d4989bd5a84b5dd0b0f5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6434d4989bd5a84b5dd0b0f5/0Elf9qbfG9Hkgypm9pTGm.jpeg
paper_authors.fullname: Dayiheng Liu
paper_authors.type: user
paper_authors.user: Losin94
paper_authors._id: 6695e0de321386ed51decbe3
paper_authors.name: Fei Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:14:50.471Z
paper_authors._id: 635b8b6a37c6a2c12e2cce00
paper_authors.avatarUrl: /avatars/229fb72180529141515d1df797b33709.svg
paper_authors.fullname: Fei Huang
paper_authors.type: user
paper_authors.user: hzhwcmhf
paper_authors._id: 6695e0de321386ed51decbe4
paper_authors.name: Guanting Dong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:15:09.576Z
paper_authors._id: 61cd4b833dd34ba1985e0753
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61cd4b833dd34ba1985e0753/BfHfrwotoMESpXZOHiIe4.png
paper_authors.fullname: KABI
paper_authors.type: user
paper_authors.user: dongguanting
paper_authors._id: 6695e0de321386ed51decbe5
paper_authors.name: Haoran Wei
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:16:15.069Z
paper_authors._id: 628b430afc0078a72e38b04a
paper_authors.avatarUrl: /avatars/b4958c184be06534645f2284635d850e.svg
paper_authors.fullname: Haoran Wei
paper_authors.type: user
paper_authors.user: whr94621
paper_authors._id: 6695e0de321386ed51decbe6
paper_authors.name: Huan Lin
paper_authors._id: 6695e0de321386ed51decbe7
paper_authors.name: Jialong Tang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:16:46.155Z
paper_authors._id: 63281d05ac205d01918b5fc7
paper_authors.avatarUrl: /avatars/fc3e0f7285bb2869a92670f764dfc535.svg
paper_authors.fullname: Jialong Tang
paper_authors.type: user
paper_authors.user: Jialong
paper_authors._id: 6695e0de321386ed51decbe8
paper_authors.name: Jialin Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-21T10:00:48.017Z
paper_authors._id: 6634979161776e1d8d35b16c
paper_authors.avatarUrl: /avatars/32a1fac0016445959c2a062c1ab76ab9.svg
paper_authors.fullname: jialinwang
paper_authors.type: user
paper_authors.user: jialinwangpku
paper_authors._id: 6695e0de321386ed51decbe9
paper_authors.name: Jian Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-14T19:05:31.557Z
paper_authors._id: 64ccb9bfead94891d12aef42
paper_authors.avatarUrl: /avatars/c54809d43d93d3f0766bd2555cecc4e3.svg
paper_authors.fullname: Yang Jian
paper_authors.type: user
paper_authors.user: CSJianYang
paper_authors._id: 6695e0de321386ed51decbea
paper_authors.name: Jianhong Tu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:18:35.197Z
paper_authors._id: 654bead777401b47e6424f88
paper_authors.avatarUrl: /avatars/7bcbdbb051c93b004f0dc3ad36c4a0ce.svg
paper_authors.fullname: Jianhong Tu
paper_authors.type: user
paper_authors.user: ToviTu
paper_authors._id: 6695e0de321386ed51decbeb
paper_authors.name: Jianwei Zhang
paper_authors._id: 6695e0de321386ed51decbec
paper_authors.name: Jianxin Ma
paper_authors._id: 6695e0de321386ed51decbed
paper_authors.name: Jin Xu
paper_authors._id: 6695e0de321386ed51decbee
paper_authors.name: Jingren Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:19:28.484Z
paper_authors._id: 602f88f5e8149a962412a667
paper_authors.avatarUrl: /avatars/b78f0e583df8e5d5e3365934fe5f4900.svg
paper_authors.fullname: Zhou
paper_authors.type: user
paper_authors.user: Jingren
paper_authors._id: 6695e0de321386ed51decbef
paper_authors.name: Jinze Bai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:19:35.233Z
paper_authors._id: 60113fad51e116b62cd0a30e
paper_authors.avatarUrl: /avatars/469357d0a4a5d2e104ae5e32801b395d.svg
paper_authors.fullname: Jinze Bai
paper_authors.type: user
paper_authors.user: Jinze
paper_authors._id: 6695e0de321386ed51decbf0
paper_authors.name: Jinzheng He
paper_authors._id: 6695e0de321386ed51decbf1
paper_authors.name: Junyang Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:19:49.891Z
paper_authors._id: 620760a26e3b7210c2ff1943
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/620760a26e3b7210c2ff1943/VC-rKqimF6yxGESNVlPoR.jpeg
paper_authors.fullname: Junyang Lin
paper_authors.type: user
paper_authors.user: JustinLin610
paper_authors._id: 6695e0de321386ed51decbf2
paper_authors.name: Kai Dang
paper_authors._id: 6695e0de321386ed51decbf3
paper_authors.name: Keming Lu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:20:12.506Z
paper_authors._id: 6453fa96ed6d7fede94408e0
paper_authors.avatarUrl: /avatars/e8c9025ef24cec958c87a1008bb54fd7.svg
paper_authors.fullname: Keming Lu
paper_authors.type: user
paper_authors.user: keminglu
paper_authors._id: 6695e0de321386ed51decbf4
paper_authors.name: Keqin Chen
paper_authors._id: 6695e0de321386ed51decbf5
paper_authors.name: Kexin Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:20:39.738Z
paper_authors._id: 65b0b3957e5d5a4ecc750de0
paper_authors.avatarUrl: /avatars/e0d79d3265ca4ad5c5411feb01043fb4.svg
paper_authors.fullname: Kexin Yang
paper_authors.type: user
paper_authors.user: dawn0929
paper_authors._id: 6695e0de321386ed51decbf6
paper_authors.name: Mei Li
paper_authors._id: 6695e0de321386ed51decbf7
paper_authors.name: Mingfeng Xue
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:20:47.889Z
paper_authors._id: 5f8946925d083370c711f296
paper_authors.avatarUrl: /avatars/14246aae3b1f8b7ad050f8ff2c8b260e.svg
paper_authors.fullname: Mingfeng Xue
paper_authors.type: user
paper_authors.user: mingfengxue
paper_authors._id: 6695e0de321386ed51decbf8
paper_authors.name: Na Ni
paper_authors._id: 6695e0de321386ed51decbf9
paper_authors.name: Pei Zhang
paper_authors._id: 6695e0de321386ed51decbfa
paper_authors.name: Peng Wang
paper_authors._id: 6695e0de321386ed51decbfb
paper_authors.name: Ru Peng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:21:03.708Z
paper_authors._id: 6687b233586426849536faff
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6687b233586426849536faff/q7EBRrWlk2eYidsKCPC9h.jpeg
paper_authors.fullname: Ru Peng
paper_authors.type: user
paper_authors.user: RuPeng
paper_authors._id: 6695e0de321386ed51decbfc
paper_authors.name: Rui Men
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-23T16:29:56.514Z
paper_authors._id: 6209bb200436d7d6f27cbeea
paper_authors.avatarUrl: /avatars/0b8a72a8b66ef7b36780fe2ccc343f78.svg
paper_authors.fullname: Iurnem
paper_authors.type: user
paper_authors.user: Iurnem
paper_authors._id: 6695e0de321386ed51decbfd
paper_authors.name: Ruize Gao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:21:20.781Z
paper_authors._id: 6629ed94aabce1b25c3db90c
paper_authors.avatarUrl: /avatars/cbc39db81c8e8f950d3bd2c2e03f71c8.svg
paper_authors.fullname: Ruize Gao
paper_authors.type: user
paper_authors.user: gaoruize
paper_authors._id: 6695e0de321386ed51decbfe
paper_authors.name: Runji Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:21:27.836Z
paper_authors._id: 649a52e5de0fb7f3f499e583
paper_authors.avatarUrl: /avatars/25f6106fa168ae57ad5cd8ef55c70d31.svg
paper_authors.fullname: Runji Lin
paper_authors.type: user
paper_authors.user: RunjiLin
paper_authors._id: 6695e0de321386ed51decbff
paper_authors.name: Shijie Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:22:16.009Z
paper_authors._id: 6472321922016353ae3ab2e9
paper_authors.avatarUrl: /avatars/93d9e397ae6079ea0672d6b54234f388.svg
paper_authors.fullname: Shijie Wang
paper_authors.type: user
paper_authors.user: simonJJJ
paper_authors._id: 6695e0de321386ed51decc00
paper_authors.name: Shuai Bai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:22:43.791Z
paper_authors._id: 63451cf0a05b51f7ded25505
paper_authors.avatarUrl: /avatars/dec4bbee4a82b773fc58dfc2dce9dbeb.svg
paper_authors.fullname: shuai bai
paper_authors.type: user
paper_authors.user: bluelike
paper_authors._id: 6695e0de321386ed51decc01
paper_authors.name: Sinan Tan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:22:58.805Z
paper_authors._id: 6337a9bb0267ebcf026ad110
paper_authors.avatarUrl: /avatars/12a170b28ade8df979067077828d719c.svg
paper_authors.fullname: Sinan Tan
paper_authors.type: user
paper_authors.user: tinytangent
paper_authors._id: 6695e0de321386ed51decc02
paper_authors.name: Tianhang Zhu
paper_authors._id: 6695e0de321386ed51decc03
paper_authors.name: Tianhao Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:23:54.179Z
paper_authors._id: 64abb96bd691b1c2482e7c19
paper_authors.avatarUrl: /avatars/ff3a429f985a52c9c5ea4f64872599f2.svg
paper_authors.fullname: litianhao
paper_authors.type: user
paper_authors.user: litianhao
paper_authors._id: 6695e0de321386ed51decc04
paper_authors.name: Tianyu Liu
paper_authors._id: 6695e0de321386ed51decc05
paper_authors.name: Wenbin Ge
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:24:46.979Z
paper_authors._id: 634d06c6f0a69955f662e641
paper_authors.avatarUrl: /avatars/5a0af8af0a21d2a93192f4a3c430fc60.svg
paper_authors.fullname: Wenbin Ge
paper_authors.type: user
paper_authors.user: gewenbin292
paper_authors._id: 6695e0de321386ed51decc06
paper_authors.name: Xiaodong Deng
paper_authors._id: 6695e0de321386ed51decc07
paper_authors.name: Xiaohuan Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-04T08:53:46.525Z
paper_authors._id: 668f5875b5b3081d776e4094
paper_authors.avatarUrl: /avatars/8c763393f25afbe5fb8b132f775e746a.svg
paper_authors.fullname: Xiaohuan Zhou
paper_authors.type: user
paper_authors.user: XiaohuanZhou
paper_authors._id: 6695e0de321386ed51decc08
paper_authors.name: Xingzhang Ren
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-03T14:06:34.985Z
paper_authors._id: 645c82c4081be4b32049633a
paper_authors.avatarUrl: /avatars/e5a08cf0ec5a04bd9d66111382ce0508.svg
paper_authors.fullname: xzhren
paper_authors.type: user
paper_authors.user: xingzhang
paper_authors._id: 6695e0de321386ed51decc09
paper_authors.name: Xinyu Zhang
paper_authors._id: 6695e0de321386ed51decc0a
paper_authors.name: Xipin Wei
paper_authors._id: 6695e0de321386ed51decc0b
paper_authors.name: Xuancheng Ren
paper_authors._id: 6695e0de321386ed51decc0c
paper_authors.name: Yang Fan
paper_authors._id: 6695e0de321386ed51decc0d
paper_authors.name: Yang Yao
paper_authors._id: 6695e0de321386ed51decc0e
paper_authors.name: Yichang Zhang
paper_authors._id: 6695e0de321386ed51decc0f
paper_authors.name: Yu Wan
paper_authors._id: 6695e0de321386ed51decc10
paper_authors.name: Yunfei Chu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:25:36.195Z
paper_authors._id: 62c6a751a71b40cf26f359a8
paper_authors.avatarUrl: /avatars/49abd2e71946035452c316d703baaac6.svg
paper_authors.fullname: Yunfei Chu
paper_authors.type: user
paper_authors.user: faychu
paper_authors._id: 6695e0de321386ed51decc11
paper_authors.name: Zeyu Cui
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-23T14:22:32.235Z
paper_authors._id: 643fb14a18afbc4d1f3ebfb4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/643fb14a18afbc4d1f3ebfb4/2qeL_qPSB9_MTzZf46ynJ.png
paper_authors.fullname: czy yente
paper_authors.type: user
paper_authors.user: cyente
paper_authors._id: 6695e0de321386ed51decc12
paper_authors.name: Zhenru Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-18T09:27:56.417Z
paper_authors._id: 64704e973601bb7b06643e98
paper_authors.avatarUrl: /avatars/52e51f4d1be6769e4397b8be2799cf32.svg
paper_authors.fullname: Zhenru Zhang
paper_authors.type: user
paper_authors.user: Zhenru
paper_authors._id: 6695e0de321386ed51decc13
paper_authors.name: Zhihao Fan
paper_title: Qwen2 Technical Report
paper_summary: This report introduces the Qwen2 series, the latest addition to our large
  language models and large multimodal models. We release a comprehensive suite
  of foundational and instruction-tuned language models, encompassing a parameter
  range from 0.5 to 72 billion, featuring dense models and a Mixture-of-Experts
  model. Qwen2 surpasses most prior open-weight models, including its predecessor
  Qwen1.5, and exhibits competitive performance relative to proprietary models
  across diverse benchmarks on language understanding, generation, multilingual
  proficiency, coding, mathematics, and reasoning.
    The flagship model, Qwen2-72B, showcases remarkable performance: 84.2 on
  MMLU, 37.9 on GPQA, 64.6 on HumanEval, 89.5 on GSM8K, and 82.4 on BBH as a base
  language model. The instruction-tuned variant, Qwen2-72B-Instruct, attains 9.1
  on MT-Bench, 48.1 on Arena-Hard, and 35.7 on LiveCodeBench. Moreover, Qwen2
  demonstrates robust multilingual capabilities, proficient in approximately 30
  languages, spanning English, Chinese, Spanish, French, German, Arabic, Russian,
  Korean, Japanese, Thai, Vietnamese, and more, underscoring its versatility and
  global reach.
    To foster community innovation and accessibility, we have made the Qwen2
  model weights openly available on Hugging Face1 and ModelScope2, and the
  supplementary materials including example code on GitHub3. These platforms also
  include resources for quantization, fine-tuning, and deployment, facilitating a
  wide range of applications and research endeavors.
paper_ai_summary: The Qwen2 series, comprising 0.5 to 72 billion parameter models, surpasses prior open models across language understanding, generation, multilingualism, coding, math, and reasoning, with exceptional performance in benchmarks like MMLU, GPQA, HumanEval, GSM8K, BBH, MT-Bench, Arena-Hard, and LiveCodeBench.
paper_ai_keywords: Mixture-of-Experts language models multimodal models MMLU GPQA HumanEval GSM8K BBH MT-Bench Arena-Hard LiveCodeBench