and this can be converted to

yc = y

(4)
and Eq-(2,3,4) indicate that, in the ﬁrst training step, all the inputs and outputs of both the trainable and
locked copy of neural network blocks are consistent with what they would be as if the ControlNet does
not exist. In other words, when a ControlNet is applied to some neural network blocks, before any
optimization, it will not cause any inﬂuence to the deep neural features. The capability, functionality,
and result quality of any neural network block is perfectly preserved, and any further optimization
will become as fast as ﬁne tuning (compared to train those layers from scratch).
We brieﬂy deduce the gradient calculation of a zero convolution layer. Considering an 1 × 1
convolution layer with weight W and bias B, at any spatial position p and channel-wise index i,
given an input map I ∈ Rh×w×c, the forward pass can be written as

Z(I;{W , B})p,i = Bi +

Ip,iWi,j

c(cid:88)

j
Adding Conditional Control to Text-to-Image Diffusion Models