13

Input (User Scribble)DefaultAutomatic PromptUser Prompt“a turtle in river”“a masterpiece of cartoon-style turtle illustration”“a cow with horns standing in a field”“a robot ox on moon, UE5 rendering, ray tracing”“a digital painting of a hot air balloon”“magic hot air balloon over a lit magic city at night”“a door on a wall”“magical door, Hearthstone”“an elephant with background in the field”“Egyptian elephant sculpture”Figure 7: Controlling Stable Diffusion with HED boundary map. The “automatic prompts” are generated by BLIP based on the default result
images without using user prompts. See also the Appendix for source images for HED boundary detection.

14
Adding Conditional Control to Text-to-Image Diffusion Models