title: Afterburner: Reinforcement Learning Facilitates Self-Improving Code
    Efficiency Optimization
summary: Large Language Models (LLMs) generate functionally correct solutions but
  often fall short in code efficiency, a critical bottleneck for real-world
  deployment. In this paper, we introduce a novel test-time iterative
  optimization framework to address this, employing a closed-loop system where
  LLMs iteratively refine code based on empirical performance feedback from an
  execution sandbox. We explore three training strategies: Supervised Fine-Tuning
  (SFT), Direct Preference Optimization (DPO), and Group Relative Policy
  Optimization~(GRPO). Experiments on our Venus dataset and the APPS benchmark
  show that SFT and DPO rapidly saturate in efficiency gains. In contrast, GRPO,
  using reinforcement learning (RL) with execution feedback, continuously
  optimizes code performance, significantly boosting both pass@1 (from 47% to
  62%) and the likelihood of outperforming human submissions in efficiency (from
  31% to 45%). Our work demonstrates effective test-time code efficiency
  improvement and critically reveals the power of RL in teaching LLMs to truly
  self-improve code efficiency.
paper_authors._id: 68397697a14020a996c30112
paper_authors.name: Mingzhe Du
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-07T05:49:51.005Z
paper_authors._id: 61711f02e0b1ddb56eb9b526
paper_authors.avatarUrl: /avatars/3e2fdf774f5bc1f73b450486d6da42d4.svg
paper_authors.fullname: Mingzhe Du
paper_authors.type: user
paper_authors.user: Elfsong
paper_authors._id: 68397697a14020a996c30113
paper_authors.name: Luu Tuan Tuan
paper_authors._id: 68397697a14020a996c30114
paper_authors.name: Yue Liu
paper_authors._id: 68397697a14020a996c30115
paper_authors.name: Yuhao Qing
paper_authors._id: 68397697a14020a996c30116
paper_authors.name: Dong Huang
paper_authors._id: 68397697a14020a996c30117
paper_authors.name: Xinyi He
paper_authors._id: 68397697a14020a996c30118
paper_authors.name: Qian Liu
paper_authors._id: 68397697a14020a996c30119
paper_authors.name: Zejun Ma
paper_authors._id: 68397697a14020a996c3011a
paper_authors.name: See-kiong Ng
paper_title: Afterburner: Reinforcement Learning Facilitates Self-Improving Code
    Efficiency Optimization
paper_summary: Large Language Models (LLMs) generate functionally correct solutions but
  often fall short in code efficiency, a critical bottleneck for real-world
  deployment. In this paper, we introduce a novel test-time iterative
  optimization framework to address this, employing a closed-loop system where
  LLMs iteratively refine code based on empirical performance feedback from an
  execution sandbox. We explore three training strategies: Supervised Fine-Tuning
  (SFT), Direct Preference Optimization (DPO), and Group Relative Policy
  Optimization~(GRPO). Experiments on our Venus dataset and the APPS benchmark
  show that SFT and DPO rapidly saturate in efficiency gains. In contrast, GRPO,
  using reinforcement learning (RL) with execution feedback, continuously
  optimizes code performance, significantly boosting both pass@1 (from 47% to
  62%) and the likelihood of outperforming human submissions in efficiency (from
  31% to 45%). Our work demonstrates effective test-time code efficiency
  improvement and critically reveals the power of RL in teaching LLMs to truly
  self-improve code efficiency.
paper_ai_summary: A novel test-time iterative optimization framework using reinforcement learning continuously enhances code efficiency generated by large language models.
paper_ai_keywords: Large Language Models iterative optimization closed-loop system empirical performance feedback Supervised Fine-Tuning Direct Preference Optimization Group Relative Policy Optimization reinforcement learning pass@1 Venus dataset APPS benchmark