Diffusion models have been popularly applied
to unsupervised (Dhariwal and Nichol, 2021) and
text conditioned image generation tasks (Saharia
et al., 2022a,b; Ramesh et al., 2021). More recently,
similar approaches have been extended to work in
discrete domains such as text (Li et al., 2022; Lin
et al., 2023; Li et al., 2023b; Reid et al., 2023).

3 Methodology

Figure 1 shows CODEFUSION’s architecture. This
section describes each component and our training
and inference procedures.

3.1 Architecture
The input to CODEFUSION is a natural language
utterance s = {s1, s2,··· , sk} and the output is a
predicted code snippet ˆy = {ˆy1, ˆy2,··· , ˆys}. Both
input and output are padded to a fixed dimension
n. CODEFUSION has three main transformer-based
components (an encoder E, a denoiser N, a de-
coder D) and a classification head H.

The transformer-based encoder transforms the
tokenized utterance s into a vector representation
Es = E(s) = {e1, e2,··· , en}.
CODEFUSION