Could you explain, how your independent and joint losses are created?
You could use something like this as a starter:
modelA = nn.Linear(10, 2).to('cuda:0')
modelB = nn.Linear(10, 2).to('cuda:1')

criterion = nn.CrossEntropyLoss()

optimizerA = optim.SGD(modelA.parameters(), lr=1e-3)
optimizerB = optim.SGD(modelB.parameters(), lr=1e-3)

for data, target in loader:
    # Get losses for separate models
    data, target = data.to('cuda:0'), target.to('cuda:0')
    output = modelA(data)
    lossA = criterion(output, target)
    
    data, target = data.to('cuda:1'), target.to('cuda:1')
    output = modelB(data)
    lossB = criterion(output, target)

I’m not sure how you would like to create the joint loss, i.e. just summing lossA and lossB wouldn’t change anything.