Test-Driven Development for Code Generation
<|reference_start|>Test-Driven Development for Code Generation: Recent Large Language Models (LLMs) have demonstrated significant capabilities in generating code snippets directly from problem statements. This increasingly automated process mirrors traditional human-led software development, where code is often written in response to a requirement. Historically, Test-Driven Development (TDD) has proven its merit, requiring developers to write tests before the functional code, ensuring alignment with the initial problem statements. Applying TDD principles to LLM-based code generation offers one distinct benefit: it enables developers to verify the correctness of generated code against predefined tests. This paper investigates if and how TDD can be incorporated into AI-assisted code-generation processes. We experimentally evaluate our hypothesis that providing LLMs like GPT-4 and Llama 3 with tests in addition to the problem statements enhances code generation outcomes. We experimented with established function-level code generation benchmarks such as MBPP and HumanEval. Our results consistently demonstrate that including test cases leads to higher success in solving programming challenges. We assert that TDD is a promising paradigm for helping ensure that the code generated by LLMs effectively captures the requirements.<|reference_end|>
@article{mathews2024test-driven,
    title={Test-Driven Development for Code Generation},
    author={Noble Saji Mathews and Meiyappan Nagappan},
    journal={arXiv preprint arXiv:2402.13521},
    year={2024},
    doi={10.1145/3691620.3695527},
    archivePrefix={arXiv},
    eprint={2402.13521},
    primaryClass={cs.SE cs.AI}
}