Yes , you are right .  But the lossB have been moved to GPU 0. I wonder that could affect gradient of modelB in backward operation . Could modelB  parameters be updated synchronously ?
Thanks for your reply again.