img_train1143×414 46.6 KB
I have a doubt regarding what’s going on behind the implementation of the architecture shown in the figure.
In the figure outputs 1 and 2 come from inputs 1 and 2 respectively.
Assume  that we detach output 1 and set requires_grad = False for F1 . It is observed empirically that loss.backward() works however based on the documentation gradient computation will stop after reaching F1 in a backward pass . So how exactly are the gradients being updated ?
Will there be any difference if we set requires_grad = True for F1 ?
Suppose now that output 1 is not detached and the loss is backpropagated this time .
Firstly , how exactly is the resultant gradient for the loss computed ?
Is it the sum of the partial derivatives we get with respect to outputs 1 and 2 or is it calculated in some other manner ?
Lastly if we set requires_grad = False for process 1 and input 1 and do the loss computation again will the gradients for output 2 have the same value as when output 1 was detached ?