I think your approach is alright. A small improvement might be to switch the order of updating the models.
You could first call loss2.backward(retain_graph=True) and update B as you need this gradient in A a bit later. Then call loss1.backward() and update A.
Here is a small example:

modelA = nn.Linear(10, 10)
modelB = nn.Linear(10, 10)

x = torch.randn(1, 10)
target = torch.randn(1, 10)

criterionA = nn.MSELoss()
criterionB = nn.L1Loss()

optimizerA = optim.Adam(modelA.parameters(), lr=1e-3)
optimizerB = optim.Adam(modelB.parameters(), lr=1e-3)

y = modelA(x)
z = modelB(y)

lossA = criterionA(z, target)
lossB = criterionB(z, target)

optimizerA.zero_grad()
optimizerB.zero_grad()

# modelB Update
lossB.backward(retain_graph=True)

# Both models now have the gradient using lossB
print(modelA.weight.grad)
print(modelB.weight.grad)

# Update modelB (lossB grad)
optimizerB.step()

# modelA Updata
lossA.backward()

# Now modelA has the accumulated gradients of lossA and lossB
print(modelA.weight.grad)

# Update modelA (lossA + lossB)
optimizerA.step()