RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning
<|reference_start|>RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning: Large language models (LLMs) deployed as agents solve user-specified tasks over multiple steps while keeping the required manual engagement to a minimum. Crucially, such LLMs need to ground their generations in any feedback obtained to reliably achieve desired outcomes. We propose an end-to-end reinforcement learning method for teaching models to leverage execution feedback in the realm of code synthesis, where state-of-the-art LLMs struggle to improve code iteratively compared to independent sampling. We benchmark on competitive programming tasks, where we achieve new start-of-the art results with both small (8B parameters) and large (70B) models while reducing the amount of samples required by an order of magnitude. Our analysis of inference-time behavior demonstrates that our method produces LLMs that effectively leverage automatic feedback over multiple steps.<|reference_end|>
@article{gehring2024rlef:,
    title={RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement
  Learning},
    author={Jonas Gehring, Kunhao Zheng, Jade Copet, Vegard Mella, Taco Cohen,
  Gabriel Synnaeve},
    journal={arXiv preprint arXiv:2410.02089},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.02089},
    primaryClass={cs.CL cs.AI}
}