Thank you very much!
So if I got that right the Solution would be as follows:
    output2, output3 = model(input)
    loss3 = criterion3(output3, label3)
    loss3.backward()
    optimizer3.step() 
    
    optimizer12.zero_grad()
    output2, output3 = model(input)
    loss2 = criterion2(output2, label2)
    loss3 = criterion3(output3, label3)
    loss12 = loss2 + some_value*loss3
    loss12.backward()
    optimizer12.step()

Which means one has to forward the input through the network a second time, in order to get the “new” activations for the shared layer.
I am not sure about the optimizer12.zero_grad(). I think before the second forward pass I will have to zero the gradients, at least for fc2, as if I would not do it the the gradients would get summed up for both forward-passes. Is that right?
Again, thanks a lot for your help!