CodeFusion: A Pre-trained Diffusion Model for Code Generation
Imagine a developer who can only change their last line of code, how often
would they have to start writing a function from scratch before it is correct?
Auto-regressive models for code generation from natural language have a similar
limitation: they do not easily allow reconsidering earlier tokens generated. We
introduce CodeFusion, a pre-trained diffusion code generation model that
addresses this limitation by iteratively denoising a complete program
conditioned on the encoded natural language. We evaluate CodeFusion on the task
of natural language to code generation for Bash, Python, and Microsoft Excel
conditional formatting (CF) rules. Experiments show that CodeFusion (75M
parameters) performs on par with state-of-the-art auto-regressive systems
(350M-175B parameters) in top-1 accuracy and outperforms them in top-3 and
top-5 accuracy due to its better balance in diversity versus quality.
653f192eec992c5c209bab70
Mukul Singh
claimed_verified
2023-10-30T13:12:07.622Z
63ea298fb1d9c4ef71eab282
/avatars/6a0d3b1678e2cf6b0356d0495530e045.svg
Mukul Singh
user
MukulSingh105
653f192eec992c5c209bab71
José Cambronero
653f192eec992c5c209bab72
Sumit Gulwani
653f192eec992c5c209bab73
Vu Le
653f192eec992c5c209bab74
Carina Negreanu
653f192eec992c5c209bab75
Gust Verbruggen
CodeFusion: A Pre-trained Diffusion Model for Code Generation
Imagine a developer who can only change their last line of code, how often
would they have to start writing a function from scratch before it is correct?
Auto-regressive models for code generation from natural language have a similar
limitation: they do not easily allow reconsidering earlier tokens generated. We
introduce CodeFusion, a pre-trained diffusion code generation model that
addresses this limitation by iteratively denoising a complete program
conditioned on the encoded natural language. We evaluate CodeFusion on the task
of natural language to code generation for Bash, Python, and Microsoft Excel
conditional formatting (CF) rules. Experiments show that CodeFusion (75M
parameters) performs on par with state-of-the-art auto-regressive systems
(350M-175B parameters) in top-1 accuracy and outperforms them in top-3 and
top-5 accuracy due to its better balance in diversity versus quality.
CodeFusion, a diffusion model for code generation, outperforms auto-regressive models in natural language to code tasks by iteratively refining the entire program.
diffusion code generation model iteratively denoising natural language to code generation Bash Python Microsoft Excel conditional formatting (CF) rules top-1 accuracy top-3 accuracy top-5 accuracy diversity versus quality