I have a joint network where the data go through two separate subnetworks and get concatenated before sending through a third network. I am just wondering how can I back propagate two losses such a way that the optimizer will update net1’s parameter by loss1 and net2’s parameters by loss2
Models could be these,
class net1(nn.Module):
    def __init__(self, input_size, hidden_size, num_classes, p=0.0):
        super().__init__()
        self.embeddings = nn.Sequential(
            nn.Linear(input_size, hidden_size),
            nn.ReLU()
        )
    def forward(self, x):
        return self.embeddings(x)

class net2(nn.Module):
    def __init__(self, input_size, hidden_size, num_classes, p=0.0):
        super().__init__()
        self.embeddings = nn.Sequential(
            nn.Linear(input_size, hidden_size),
            nn.ReLU()
        )
    def forward(self, x):
        return self.embeddings(x)

Training loop:
loss1 = net1(data)
loss2 = net2(data)
loss = loss1 + loss2
loss.backward()
optimizer.step()

I am just wondering, would that make sure net1 would be updated by ‘loss1’ and net2 by ‘loss2’?
or it will get a common updates from ‘loss’
Thanks in advance