LDM [72]
0.31 ± 0.09

PITI [89]
0.26 ± 0.16

ControlNet-lite
0.32 ± 0.12

ControlNet
0.35 ± 0.14

Table 2: Evaluation of semantic segmentation label recon-
struction (ADE20K) with Intersection over Union (IoU ↑).

all 4 settings. The lightweight ControlNet-lite (Figure 8c) is
not strong enough to interpret the conditioning images and
fails in the insufficient and no prompt conditions. When zero
convolutions are replaced, the performance of ControlNet
drops to about the same as ControlNet-lite, indicating that
the pretrained backbone of the trainable copy is destroyed
during finetuning (Figure 8b).

4.3. Quantitative Evaluation
AddingConditionalControltoText-to-ImageDiffusionModels