title: Kimi k1.5: Scaling Reinforcement Learning with LLMs
summary: Language model pretraining with next token prediction has proved effective
  for scaling compute but is limited to the amount of available training data.
  Scaling reinforcement learning (RL) unlocks a new axis for the continued
  improvement of artificial intelligence, with the promise that large language
  models (LLMs) can scale their training data by learning to explore with
  rewards. However, prior published work has not produced competitive results. In
  light of this, we report on the training practice of Kimi k1.5, our latest
  multi-modal LLM trained with RL, including its RL training techniques,
  multi-modal data recipes, and infrastructure optimization. Long context scaling
  and improved policy optimization methods are key ingredients of our approach,
  which establishes a simplistic, effective RL framework without relying on more
  complex techniques such as Monte Carlo tree search, value functions, and
  process reward models. Notably, our system achieves state-of-the-art reasoning
  performance across multiple benchmarks and modalities -- e.g., 77.5 on AIME,
  96.2 on MATH 500, 94-th percentile on Codeforces, 74.9 on MathVista -- matching
  OpenAI's o1. Moreover, we present effective long2short methods that use
  long-CoT techniques to improve short-CoT models, yielding state-of-the-art
  short-CoT reasoning results -- e.g., 60.8 on AIME, 94.6 on MATH500, 47.3 on
  LiveCodeBench -- outperforming existing short-CoT models such as GPT-4o and
  Claude Sonnet 3.5 by a large margin (up to +550%).
paper_authors._id: 6791b6029e215712a7cf700a
paper_authors.name: Kimi Team
paper_authors._id: 6791b6029e215712a7cf700b
paper_authors.name: Angang Du
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-08T09:00:06.815Z
paper_authors._id: 617798384ce8f8cb2c2c74a3
paper_authors.avatarUrl: /avatars/7e079f2bb6857a462f45b7449510398d.svg
paper_authors.fullname: Ang
paper_authors.type: user
paper_authors.user: duang
paper_authors._id: 6791b6029e215712a7cf700c
paper_authors.name: Bofei Gao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-23T09:41:32.895Z
paper_authors._id: 65ae21adabf6d1ccb795e9a4
paper_authors.avatarUrl: /avatars/b5dced62c6a3564095a8fa0959bc06cb.svg
paper_authors.fullname: Bofei Gao
paper_authors.type: user
paper_authors.user: KbsdJames
paper_authors._id: 6791b6029e215712a7cf700d
paper_authors.name: Bowei Xing
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-23T09:41:55.675Z
paper_authors._id: 67503f270dfe827c4068a408
paper_authors.avatarUrl: /avatars/4591c8229c7815bfd6dc4b98aea85ca8.svg
paper_authors.fullname: Bowei Xing
paper_authors.type: user
paper_authors.user: xingbowei
paper_authors._id: 6791b6029e215712a7cf700e
paper_authors.name: Changjiu Jiang
paper_authors._id: 6791b6029e215712a7cf700f
paper_authors.name: Cheng Chen
paper_authors._id: 6791b6029e215712a7cf7010
paper_authors.name: Cheng Li
paper_authors._id: 6791b6029e215712a7cf7011
paper_authors.name: Chenjun Xiao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-23T09:42:24.770Z
paper_authors._id: 66c9aa8fa40c9235cb446365
paper_authors.avatarUrl: /avatars/616719ed848e1f8437fb39e6184feb52.svg
paper_authors.fullname: Chenjun Xiao
paper_authors.type: user
paper_authors.user: shelowize
paper_authors._id: 6791b6029e215712a7cf7012
paper_authors.name: Chenzhuang Du
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-23T09:42:36.347Z
paper_authors._id: 64c21fb42426d683e56b42bf
paper_authors.avatarUrl: /avatars/60359fe204e32af831d701d2975c4599.svg
paper_authors.fullname: Du
paper_authors.type: user
paper_authors.user: DuChenZhuang
paper_authors._id: 6791b6029e215712a7cf7013
paper_authors.name: Chonghua Liao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-23T09:43:07.678Z
paper_authors._id: 667fdaee20ee9ac417c7708c
paper_authors.avatarUrl: /avatars/69dfba6ff392643af1dcfe8af0a42ae9.svg
paper_authors.fullname: Chonghua Liao
paper_authors.type: user
paper_authors.user: ChonghuaLiao
paper_authors._id: 6791b6029e215712a7cf7014
paper_authors.name: Chuning Tang
paper_authors._id: 6791b6029e215712a7cf7015
paper_authors.name: Congcong Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-23T09:43:24.138Z
paper_authors._id: 5eefd87c5e979253a010eee5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1603575136094-5eefd87c5e979253a010eee5.jpeg
paper_authors.fullname: Congcong Wang
paper_authors.type: user
paper_authors.user: congcongwang
paper_authors._id: 6791b6029e215712a7cf7016
paper_authors.name: Dehao Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-23T09:44:01.384Z
paper_authors._id: 634d0eeffc4873ed0d00406a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1665994473058-noauth.jpeg
paper_authors.fullname: ZhangDehao
paper_authors.type: user
paper_authors.user: ispoon
paper_authors._id: 6791b6029e215712a7cf7017
paper_authors.name: Enming Yuan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-23T09:44:09.134Z
paper_authors._id: 6331606f18711776b4655e67
paper_authors.avatarUrl: /avatars/1479c2ca743b9f92d845b0ed23fcd07b.svg
paper_authors.fullname: Enming Yuan
paper_authors.type: user
paper_authors.user: EnmingYuan
paper_authors._id: 6791b6029e215712a7cf7018
paper_authors.name: Enzhe Lu
paper_authors._id: 6791b6029e215712a7cf7019
paper_authors.name: Fengxiang Tang
paper_authors._id: 6791b6029e215712a7cf701a
paper_authors.name: Flood Sung
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-23T09:44:34.078Z
paper_authors._id: 6343d01a08c017b2c042305d
paper_authors.avatarUrl: /avatars/790c4104d80da9887d481f9efb494d81.svg
paper_authors.fullname: Flood Sung
paper_authors.type: user
paper_authors.user: floodsung
paper_authors._id: 6791b6029e215712a7cf701b
paper_authors.name: Guangda Wei
paper_authors._id: 6791b6029e215712a7cf701c
paper_authors.name: Guokun Lai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-23T09:44:52.466Z
paper_authors._id: 63b4c71758f367a212c4f9ef
paper_authors.avatarUrl: /avatars/d61736e0ae8b333a7c24eb411378698c.svg
paper_authors.fullname: Lai
paper_authors.type: user
paper_authors.user: Guokun
paper_authors._id: 6791b6029e215712a7cf701d
paper_authors.name: Haiqing Guo
paper_authors._id: 6791b6029e215712a7cf701e
paper_authors.name: Han Zhu
paper_authors._id: 6791b6029e215712a7cf701f
paper_authors.name: Hao Ding
paper_authors._id: 6791b6029e215712a7cf7020
paper_authors.name: Hao Hu
paper_authors._id: 6791b6029e215712a7cf7021
paper_authors.name: Hao Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-23T15:05:07.301Z
paper_authors._id: 64ec364e7e2ec711a7601cde
paper_authors.avatarUrl: /avatars/6ba47d496586de65df183f056d35982b.svg
paper_authors.fullname: Hao Yang
paper_authors.type: user
paper_authors.user: hayayanghao
paper_authors._id: 6791b6029e215712a7cf7022
paper_authors.name: Hao Zhang
paper_authors._id: 6791b6029e215712a7cf7023
paper_authors.name: Haotian Yao
paper_authors._id: 6791b6029e215712a7cf7024
paper_authors.name: Haotian Zhao
paper_authors._id: 6791b6029e215712a7cf7025
paper_authors.name: Haoyu Lu
paper_authors._id: 6791b6029e215712a7cf7026
paper_authors.name: Haoze Li
paper_authors._id: 6791b6029e215712a7cf7027
paper_authors.name: Haozhen Yu
paper_authors._id: 6791b6029e215712a7cf7028
paper_authors.name: Hongcheng Gao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-23T09:47:40.340Z
paper_authors._id: 62728f4f6253fe2068da1021
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62728f4f6253fe2068da1021/KZ65X0EH98AF3zXemPiap.jpeg
paper_authors.fullname: Hongcheng Gao
paper_authors.type: user
paper_authors.user: HongchengGao
paper_authors._id: 6791b6029e215712a7cf7029
paper_authors.name: Huabin Zheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-23T09:47:47.850Z
paper_authors._id: 61860e1258cb1f8c362f9441
paper_authors.avatarUrl: /avatars/8dbc8209ad0d918453c1ffacc8f61e7f.svg
paper_authors.fullname: Huabin Zheng
paper_authors.type: user
paper_authors.user: zhenghuabin
paper_authors._id: 6791b6029e215712a7cf702a
paper_authors.name: Huan Yuan
paper_authors._id: 6791b6029e215712a7cf702b
paper_authors.name: Jia Chen
paper_authors._id: 6791b6029e215712a7cf702c
paper_authors.name: Jianhang Guo
paper_authors._id: 6791b6029e215712a7cf702d
paper_authors.name: Jianlin Su
paper_authors._id: 6791b6029e215712a7cf702e
paper_authors.name: Jianzhou Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-23T09:48:00.137Z
paper_authors._id: 63be6bf6da08ed0544f1eb7a
paper_authors.avatarUrl: /avatars/19b5be6d3296da402d8822e51d6376e2.svg
paper_authors.fullname: jianzhouWang
paper_authors.type: user
paper_authors.user: jianzhouWang
paper_authors._id: 6791b6029e215712a7cf702f
paper_authors.name: Jie Zhao
paper_authors._id: 6791b6029e215712a7cf7030
paper_authors.name: Jin Zhang
paper_authors._id: 6791b6029e215712a7cf7031
paper_authors.name: Jingyuan Liu
paper_authors._id: 6791b6029e215712a7cf7032
paper_authors.name: Junjie Yan
paper_authors._id: 6791b6029e215712a7cf7033
paper_authors.name: Junyan Wu
paper_authors._id: 6791b6029e215712a7cf7034
paper_authors.name: Lidong Shi
paper_authors._id: 6791b6029e215712a7cf7035
paper_authors.name: Ling Ye
paper_authors._id: 6791b6029e215712a7cf7036
paper_authors.name: Longhui Yu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-24T10:01:45.720Z
paper_authors._id: 64b753306c169983c982f609
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64b753306c169983c982f609/oOC_N8cI1Go6avUMvIPVI.jpeg
paper_authors.fullname: Longhui Yu
paper_authors.type: user
paper_authors.user: Longhui98
paper_authors._id: 6791b6029e215712a7cf7037
paper_authors.name: Mengnan Dong
paper_authors._id: 6791b6029e215712a7cf7038
paper_authors.name: Neo Zhang
paper_authors._id: 6791b6029e215712a7cf7039
paper_authors.name: Ningchen Ma
paper_authors._id: 6791b6029e215712a7cf703a
paper_authors.name: Qiwei Pan
paper_authors._id: 6791b6029e215712a7cf703b
paper_authors.name: Qucheng Gong
paper_authors._id: 6791b6029e215712a7cf703c
paper_authors.name: Shaowei Liu
paper_authors._id: 6791b6029e215712a7cf703d
paper_authors.name: Shengling Ma
paper_authors._id: 6791b6029e215712a7cf703e
paper_authors.name: Shupeng Wei
paper_authors._id: 6791b6029e215712a7cf703f
paper_authors.name: Sihan Cao
paper_authors._id: 6791b6029e215712a7cf7040
paper_authors.name: Siying Huang
paper_authors._id: 6791b6029e215712a7cf7041
paper_authors.name: Tao Jiang
paper_authors._id: 6791b6029e215712a7cf7042
paper_authors.name: Weihao Gao
paper_authors._id: 6791b6029e215712a7cf7043
paper_authors.name: Weimin Xiong
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-18T09:35:12.836Z
paper_authors._id: 6225a9983207dfc568407204
paper_authors.avatarUrl: /avatars/c970db6232d84ae8c0fa5f11d561d67c.svg
paper_authors.fullname: xwm
paper_authors.type: user
paper_authors.user: xwm
paper_authors._id: 6791b6029e215712a7cf7044
paper_authors.name: Weiran He
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-15T08:04:02.280Z
paper_authors._id: 63a41c1f412fd71fb7ec4a20
paper_authors.avatarUrl: /avatars/d6837b0dbbf3507896e98754d3f8a468.svg
paper_authors.fullname: Wayne Ho
paper_authors.type: user
paper_authors.user: hewr2010
paper_authors._id: 6791b6029e215712a7cf7045
paper_authors.name: Weixiao Huang
paper_authors._id: 6791b6029e215712a7cf7046
paper_authors.name: Wenhao Wu
paper_authors._id: 6791b6029e215712a7cf7047
paper_authors.name: Wenyang He
paper_authors._id: 6791b6029e215712a7cf7048
paper_authors.name: Xianghui Wei
paper_authors._id: 6791b6029e215712a7cf7049
paper_authors.name: Xianqing Jia
paper_authors._id: 6791b6029e215712a7cf704a
paper_authors.name: Xingzhe Wu
paper_authors._id: 6791b6029e215712a7cf704b
paper_authors.name: Xinran Xu
paper_authors._id: 6791b6029e215712a7cf704c
paper_authors.name: Xinxing Zu
paper_authors._id: 6791b6029e215712a7cf704d
paper_authors.name: Xinyu Zhou
paper_authors._id: 6791b6029e215712a7cf704e
paper_authors.name: Xuehai Pan
paper_authors._id: 6791b6029e215712a7cf704f
paper_authors.name: Y. Charles
paper_authors._id: 6791b6029e215712a7cf7050
paper_authors.name: Yang Li
paper_authors._id: 6791b6029e215712a7cf7051
paper_authors.name: Yangyang Hu
paper_authors._id: 6791b6029e215712a7cf7052
paper_authors.name: Yangyang Liu
paper_authors._id: 6791b6029e215712a7cf7053
paper_authors.name: Yanru Chen
paper_authors._id: 6791b6029e215712a7cf7054
paper_authors.name: Yejie Wang
paper_authors._id: 6791b6029e215712a7cf7055
paper_authors.name: Yibo Liu
paper_authors._id: 6791b6029e215712a7cf7056
paper_authors.name: Yidao Qin
paper_authors._id: 6791b6029e215712a7cf7057
paper_authors.name: Yifeng Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-19T09:05:48.369Z
paper_authors._id: 653d276681f52ceb4d12bd85
paper_authors.avatarUrl: /avatars/56601a25e5f883a8f6dc15f6fd9dcc57.svg
paper_authors.fullname: Yifeng Liu
paper_authors.type: user
paper_authors.user: Lewis-Lau
paper_authors._id: 6791b6029e215712a7cf7058
paper_authors.name: Ying Yang
paper_authors._id: 6791b6029e215712a7cf7059
paper_authors.name: Yiping Bao
paper_authors._id: 6791b6029e215712a7cf705a
paper_authors.name: Yulun Du
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-23T09:47:18.988Z
paper_authors._id: 6340f31fb78ed99eab04ce33
paper_authors.avatarUrl: /avatars/2e7fcbf0233bdc0bc9a3f4603fd8bf90.svg
paper_authors.fullname: Du
paper_authors.type: user
paper_authors.user: Yulun
paper_authors._id: 6791b6029e215712a7cf705b
paper_authors.name: Yuxin Wu
paper_authors._id: 6791b6029e215712a7cf705c
paper_authors.name: Yuzhi Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-23T09:46:54.213Z
paper_authors._id: 67127a470a82509269d738ae
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/M9qLmI3P6dT2FIwEPFJq0.png
paper_authors.fullname: yuzhi wang
paper_authors.type: user
paper_authors.user: vin-tage
paper_authors._id: 6791b6029e215712a7cf705d
paper_authors.name: Zaida Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-23T09:46:46.940Z
paper_authors._id: 64409d69518271b0d1c033a6
paper_authors.avatarUrl: /avatars/c79eb36c4ad96286afda834e260a1c09.svg
paper_authors.fullname: zhouzaida
paper_authors.type: user
paper_authors.user: zhouzaida
paper_authors._id: 6791b6029e215712a7cf705e
paper_authors.name: Zhaoji Wang
paper_authors._id: 6791b6029e215712a7cf705f
paper_authors.name: Zhaowei Li
paper_authors._id: 6791b6029e215712a7cf7060
paper_authors.name: Zhen Zhu
paper_authors._id: 6791b6029e215712a7cf7061
paper_authors.name: Zheng Zhang
paper_authors._id: 6791b6029e215712a7cf7062
paper_authors.name: Zhexu Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-23T09:46:16.399Z
paper_authors._id: 645b8bfd78730bcc103f9757
paper_authors.avatarUrl: /avatars/1f9b7ec00744708ab609b1c4db2c2fcc.svg
paper_authors.fullname: Wang Zhexuan
paper_authors.type: user
paper_authors.user: longmao14
paper_authors._id: 6791b6029e215712a7cf7063
paper_authors.name: Zhilin Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-23T09:46:04.605Z
paper_authors._id: 64bf74154d2052b1aa5ca6d9
paper_authors.avatarUrl: /avatars/7aa6f2952cdbc20cfa758fdd905f06a6.svg
paper_authors.fullname: ZHILIN YANG
paper_authors.type: user
paper_authors.user: bruceyannnn
paper_authors._id: 6791b6029e215712a7cf7064
paper_authors.name: Zhiqi Huang
paper_authors._id: 6791b6029e215712a7cf7065
paper_authors.name: Zihao Huang
paper_authors._id: 6791b6029e215712a7cf7066
paper_authors.name: Ziyao Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-23T09:45:24.566Z
paper_authors._id: 64d381c3527b22502ec81eb7
paper_authors.avatarUrl: /avatars/55145a716acf120a32a302830c1e8178.svg
paper_authors.fullname: Ziyao Xu
paper_authors.type: user
paper_authors.user: xzyxzy
paper_authors._id: 6791b6029e215712a7cf7067
paper_authors.name: Zonghan Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-23T09:45:16.573Z
paper_authors._id: 635795f24fba057662cdf2e7
paper_authors.avatarUrl: /avatars/07642c7cfd9b57a0f8382268b009cf6f.svg
paper_authors.fullname: Zonghan Yang
paper_authors.type: user
paper_authors.user: minicheshire
paper_title: Kimi k1.5: Scaling Reinforcement Learning with LLMs
paper_summary: Language model pretraining with next token prediction has proved effective
  for scaling compute but is limited to the amount of available training data.
  Scaling reinforcement learning (RL) unlocks a new axis for the continued
  improvement of artificial intelligence, with the promise that large language
  models (LLMs) can scale their training data by learning to explore with
  rewards. However, prior published work has not produced competitive results. In
  light of this, we report on the training practice of Kimi k1.5, our latest
  multi-modal LLM trained with RL, including its RL training techniques,
  multi-modal data recipes, and infrastructure optimization. Long context scaling
  and improved policy optimization methods are key ingredients of our approach,
  which establishes a simplistic, effective RL framework without relying on more
  complex techniques such as Monte Carlo tree search, value functions, and
  process reward models. Notably, our system achieves state-of-the-art reasoning
  performance across multiple benchmarks and modalities -- e.g., 77.5 on AIME,
  96.2 on MATH 500, 94-th percentile on Codeforces, 74.9 on MathVista -- matching
  OpenAI's o1. Moreover, we present effective long2short methods that use
  long-CoT techniques to improve short-CoT models, yielding state-of-the-art
  short-CoT reasoning results -- e.g., 60.8 on AIME, 94.6 on MATH500, 47.3 on
  LiveCodeBench -- outperforming existing short-CoT models such as GPT-4o and
  Claude Sonnet 3.5 by a large margin (up to +550%).
paper_ai_summary: A multi-modal LLM trained with reinforcement learning achieves state-of-the-art reasoning performance across various benchmarks by utilizing long context scaling and effective policy optimization methods.
paper_ai_keywords: next token prediction reinforcement learning large language models multi-modal LLM RL training techniques multi-modal data recipes infrastructure optimization long context scaling policy optimization Monte Carlo tree search value functions process reward models AIME MATH 500 Codeforces MathVista long-CoT techniques short-CoT models short-CoT reasoning