Thanks in advance .
modelA = nn.Linear(10, 2).to('cuda:0')
modelB = nn.Linear(10, 2).to('cuda:1')

criterion = nn.CrossEntropyLoss()

optimizerA = optim.SGD(modelA.parameters(), lr=1e-3)
optimizerB = optim.SGD(modelB.parameters(), lr=1e-3)

for ((dataA, targetA), (dataB, targetB)) in zip(loader_A, loader_B):
    # Get losses for separate models
    dataA, targetA = dataA.to('cuda:0'), targetA.to('cuda:0')
    outputA = modelA(dataA)
    lossA = criterion(outputA, targetA)
    
    dataB, targetB = dataB.to('cuda:1'), targetB.to('cuda:1')
    outputB = modelB(dataB)
    lossB = criterion(output, target)

    lossC = torch.nn.CosineSimilarity(ouputA, outputB)
 
    final_loss = lossA + lossB + lossC

In above snippet, we aim to eliminate the discrepancy in between two independent systems. Therefore, we should jointly train them .