title: PanGu-Coder2: Boosting Large Language Models for Code with Ranking
    Feedback
summary: Large Language Models for Code (Code LLM) are flourishing. New and powerful
  models are released on a weekly basis, demonstrating remarkable performance on
  the code generation task. Various approaches have been proposed to boost the
  code generation performance of pre-trained Code LLMs, such as supervised
  fine-tuning, instruction tuning, reinforcement learning, etc. In this paper, we
  propose a novel RRTF (Rank Responses to align Test&Teacher Feedback) framework,
  which can effectively and efficiently boost pre-trained large language models
  for code generation. Under this framework, we present PanGu-Coder2, which
  achieves 62.20% pass@1 on the OpenAI HumanEval benchmark. Furthermore, through
  an extensive evaluation on CoderEval and LeetCode benchmarks, we show that
  PanGu-Coder2 consistently outperforms all previous Code LLMs.
paper_authors._id: 64c316dbf33ee56c8067528d
paper_authors.name: Bo Shen
paper_authors._id: 64c316dbf33ee56c8067528e
paper_authors.name: Jiaxin Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-29T17:45:55.917Z
paper_authors._id: 62610bf040e04009e810357c
paper_authors.avatarUrl: /avatars/16056719d57d4fcf7c3d21135e65f1a2.svg
paper_authors.fullname: Jiaxin Zhang
paper_authors.type: user
paper_authors.user: BerryBiscuits
paper_authors._id: 64c316dbf33ee56c8067528f
paper_authors.name: Taihong Chen
paper_authors._id: 64c316dbf33ee56c80675290
paper_authors.name: Daoguang Zan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-09-04T07:32:24.288Z
paper_authors._id: 61527edf8b55dbdae72874fa
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61527edf8b55dbdae72874fa/ZGWSBf_KSrDof6WyMoDMU.jpeg
paper_authors.fullname: Daoguang Zan
paper_authors.type: user
paper_authors.user: Daoguang
paper_authors._id: 64c316dbf33ee56c80675291
paper_authors.name: Bing Geng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-29T17:45:59.413Z
paper_authors._id: 646ec43b8d316fde87b3871a
paper_authors.avatarUrl: /avatars/195cc396e160bb65ef051a2e5b6a0ebc.svg
paper_authors.fullname: Geng Bing
paper_authors.type: user
paper_authors.user: edwingeng
paper_authors._id: 64c316dbf33ee56c80675292
paper_authors.name: An Fu
paper_authors._id: 64c316dbf33ee56c80675293
paper_authors.name: Muhan Zeng
paper_authors._id: 64c316dbf33ee56c80675294
paper_authors.name: Ailun Yu
paper_authors._id: 64c316dbf33ee56c80675295
paper_authors.name: Jichuan Ji
paper_authors._id: 64c316dbf33ee56c80675296
paper_authors.name: Jingyang Zhao
paper_authors._id: 64c316dbf33ee56c80675297
paper_authors.name: Yuenan Guo
paper_authors._id: 64c316dbf33ee56c80675298
paper_authors.name: Qianxiang Wang
paper_title: PanGu-Coder2: Boosting Large Language Models for Code with Ranking
    Feedback
paper_summary: Large Language Models for Code (Code LLM) are flourishing. New and powerful
  models are released on a weekly basis, demonstrating remarkable performance on
  the code generation task. Various approaches have been proposed to boost the
  code generation performance of pre-trained Code LLMs, such as supervised
  fine-tuning, instruction tuning, reinforcement learning, etc. In this paper, we
  propose a novel RRTF (Rank Responses to align Test&Teacher Feedback) framework,
  which can effectively and efficiently boost pre-trained large language models
  for code generation. Under this framework, we present PanGu-Coder2, which
  achieves 62.20% pass@1 on the OpenAI HumanEval benchmark. Furthermore, through
  an extensive evaluation on CoderEval and LeetCode benchmarks, we show that
  PanGu-Coder2 consistently outperforms all previous Code LLMs.
paper_ai_summary: PanGu-Coder2, a model fine-tuned with RRTF framework, demonstrates superior code generation performance, outperforming existing Code LLMs on multiple benchmarks.
paper_ai_keywords: RRTF pre-trained large language models code generation OpenAI HumanEval CoderEval LeetCode benchmarks