title: Training Language Models to Generate Quality Code with Program Analysis
    Feedback
summary: Code generation with large language models (LLMs), often termed vibe coding,
  is increasingly adopted in production but fails to ensure code quality,
  particularly in security (e.g., SQL injection vulnerabilities) and
  maintainability (e.g., missing type annotations). Existing methods, such as
  supervised fine-tuning and rule-based post-processing, rely on labor-intensive
  annotations or brittle heuristics, limiting their scalability and
  effectiveness. We propose REAL, a reinforcement learning framework that
  incentivizes LLMs to generate production-quality code using program
  analysis-guided feedback. Specifically, REAL integrates two automated signals:
  (1) program analysis detecting security or maintainability defects and (2) unit
  tests ensuring functional correctness. Unlike prior work, our framework is
  prompt-agnostic and reference-free, enabling scalable supervision without
  manual intervention. Experiments across multiple datasets and model scales
  demonstrate that REAL outperforms state-of-the-art methods in simultaneous
  assessments of functionality and code quality. Our work bridges the gap between
  rapid prototyping and production-ready code, enabling LLMs to deliver both
  speed and quality.
paper_authors._id: 68409fbfeb249b555b0e4c60
paper_authors.name: Feng Yao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-05T07:27:28.155Z
paper_authors._id: 64c8b2c5c547ed5243e14a6e
paper_authors.avatarUrl: /avatars/96d4a9010f96001c8cff235915926390.svg
paper_authors.fullname: Feng Yao
paper_authors.type: user
paper_authors.user: fengyao1909
paper_authors._id: 68409fbfeb249b555b0e4c61
paper_authors.name: Zilong Wang
paper_authors._id: 68409fbfeb249b555b0e4c62
paper_authors.name: Liyuan Liu
paper_authors._id: 68409fbfeb249b555b0e4c63
paper_authors.name: Junxia Cui
paper_authors._id: 68409fbfeb249b555b0e4c64
paper_authors.name: Li Zhong
paper_authors._id: 68409fbfeb249b555b0e4c65
paper_authors.name: Xiaohan Fu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-05T07:27:30.320Z
paper_authors._id: 661ad1d2e7b0ab12bcabf85c
paper_authors.avatarUrl: /avatars/d858ed936e13d352990920046ea34c19.svg
paper_authors.fullname: Xiaohan Fu
paper_authors.type: user
paper_authors.user: x5fu
paper_authors._id: 68409fbfeb249b555b0e4c66
paper_authors.name: Haohui Mai
paper_authors._id: 68409fbfeb249b555b0e4c67
paper_authors.name: Vish Krishnan
paper_authors._id: 68409fbfeb249b555b0e4c68
paper_authors.name: Jianfeng Gao
paper_authors._id: 68409fbfeb249b555b0e4c69
paper_authors.name: Jingbo Shang
paper_title: Training Language Models to Generate Quality Code with Program Analysis
    Feedback
paper_summary: Code generation with large language models (LLMs), often termed vibe coding,
  is increasingly adopted in production but fails to ensure code quality,
  particularly in security (e.g., SQL injection vulnerabilities) and
  maintainability (e.g., missing type annotations). Existing methods, such as
  supervised fine-tuning and rule-based post-processing, rely on labor-intensive
  annotations or brittle heuristics, limiting their scalability and
  effectiveness. We propose REAL, a reinforcement learning framework that
  incentivizes LLMs to generate production-quality code using program
  analysis-guided feedback. Specifically, REAL integrates two automated signals:
  (1) program analysis detecting security or maintainability defects and (2) unit
  tests ensuring functional correctness. Unlike prior work, our framework is
  prompt-agnostic and reference-free, enabling scalable supervision without
  manual intervention. Experiments across multiple datasets and model scales
  demonstrate that REAL outperforms state-of-the-art methods in simultaneous
  assessments of functionality and code quality. Our work bridges the gap between
  rapid prototyping and production-ready code, enabling LLMs to deliver both
  speed and quality.
paper_ai_summary: A reinforcement learning framework improves code quality in large language models by using automated feedback from program analysis and unit tests.
paper_ai_keywords: reinforcement learning large language models code generation program analysis unit tests prompt-agnostic reference-free