Hi, I am rather new to pytorch and working on a project. It comes to a point when I need to manipulate the gradients of the network. My question is:

Picture1.png729×378 4.4 KB

I have a network with two supervisons. Usually, I use
loss1 = alpha*loss1
loss1.backward()
loss2 =  (1-alpha)*loss2
loss2.backward()

However, if I understand this correctly, this seems to do the backpropagation twice on the network (the part before the interface). Is there any way to do something like:
#pseudocode
alpha*loss1.backward() till interface, save as error gradients1
(1-alpha)*loss2.backward() till interface, save as error gradients2
gradient = error gradients1 + error gradients2
resume the backprop on the network