Our experiments show that ControlNet can control Sta-
ble Diffusion with various conditioning inputs, including
Canny edges, Hough lines, user scribbles, human key points,
segmentation maps, shape normals, depths, etc. (Figure 1).
We test our approach using a single conditioning image,
with or without text prompts, and we demonstrate how our
approach supports the composition of multiple conditions.
Additionally, we report that the training of ControlNet is
robust and scalable on datasets of different sizes, and that for
some tasks like depth-to-image conditioning, training Con-
trolNets on a single NVIDIA RTX 3090Ti GPU can achieve

results competitive with industrial models trained on large
computation clusters. Finally, we conduct ablative studies to
investigate the contribution of each component of our model,
and compare our models to several strong conditional image
generation baselines with user studies.
AddingConditionalControltoText-to-ImageDiffusionModels