Table 3: % of top-1 generations that are syntactically-
valid for CODEFUSION and text diffusion-based base-
lines. CODEFUSION generates more valid candidates.

System
Diffusion-LM
GENIE
CODEFUSION

Python Bash CF Rules

19.5
24.2
67.6

40.4
54.2
73.4

74.3
78.6
94.5

5.2 Ablations (Q2)
Table-4 shows the results of CODEFUSION with
various changes. Removing either pre-training task
significantly reduces performance (-10.9% for code
generation and -4.6% for CPD on average across
the three languages). Results by replacing D and
H with grounding (pick closest vocabulary token
at last denoising step) or clamping (pick closest
vocabulary token at each denoising step) highlights
the benefit of using a decoder before rounding.

Table 4: Ablations for CODEFUSION (M). “–” means the
component is removed. Full model beats all ablations.

Model
M – Code Generation
M – CPD Objective
M with Grounding
M with Clamping
M (CODEFUSION)

Python Bash CF Rule

70.9
76.7
73.5
77.1
80.7
CODEFUSION