title: CodeFusion: A Pre-trained Diffusion Model for Code Generation
summary: Imagine a developer who can only change their last line of code, how often
  would they have to start writing a function from scratch before it is correct?
  Auto-regressive models for code generation from natural language have a similar
  limitation: they do not easily allow reconsidering earlier tokens generated. We
  introduce CodeFusion, a pre-trained diffusion code generation model that
  addresses this limitation by iteratively denoising a complete program
  conditioned on the encoded natural language. We evaluate CodeFusion on the task
  of natural language to code generation for Bash, Python, and Microsoft Excel
  conditional formatting (CF) rules. Experiments show that CodeFusion (75M
  parameters) performs on par with state-of-the-art auto-regressive systems
  (350M-175B parameters) in top-1 accuracy and outperforms them in top-3 and
  top-5 accuracy due to its better balance in diversity versus quality.
paper_authors._id: 653f192eec992c5c209bab70
paper_authors.name: Mukul Singh
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-30T13:12:07.622Z
paper_authors._id: 63ea298fb1d9c4ef71eab282
paper_authors.avatarUrl: /avatars/6a0d3b1678e2cf6b0356d0495530e045.svg
paper_authors.fullname: Mukul Singh
paper_authors.type: user
paper_authors.user: MukulSingh105
paper_authors._id: 653f192eec992c5c209bab71
paper_authors.name: José Cambronero
paper_authors._id: 653f192eec992c5c209bab72
paper_authors.name: Sumit Gulwani
paper_authors._id: 653f192eec992c5c209bab73
paper_authors.name: Vu Le
paper_authors._id: 653f192eec992c5c209bab74
paper_authors.name: Carina Negreanu
paper_authors._id: 653f192eec992c5c209bab75
paper_authors.name: Gust Verbruggen
paper_title: CodeFusion: A Pre-trained Diffusion Model for Code Generation
paper_summary: Imagine a developer who can only change their last line of code, how often
  would they have to start writing a function from scratch before it is correct?
  Auto-regressive models for code generation from natural language have a similar
  limitation: they do not easily allow reconsidering earlier tokens generated. We
  introduce CodeFusion, a pre-trained diffusion code generation model that
  addresses this limitation by iteratively denoising a complete program
  conditioned on the encoded natural language. We evaluate CodeFusion on the task
  of natural language to code generation for Bash, Python, and Microsoft Excel
  conditional formatting (CF) rules. Experiments show that CodeFusion (75M
  parameters) performs on par with state-of-the-art auto-regressive systems
  (350M-175B parameters) in top-1 accuracy and outperforms them in top-3 and
  top-5 accuracy due to its better balance in diversity versus quality.
paper_ai_keywords: diffusion code generation model iteratively denoising natural language to code generation Bash Python Microsoft Excel conditional formatting (CF) rules top-1 accuracy top-3 accuracy top-5 accuracy diversity versus quality