Thank you, @mrshenli . I change all self.device to something like x.device on forwarding.
… It looks like works but still, stuck after loss.backward() .
All loss calculated and, loss tensor is on cuda:0, which is default output device.
loss_lm calculated
loss_lm : 72.27577209472656
loss_lm tensor device: cuda:0, shape: torch.Size([])
after mean loss_lm : 72.27577209472656
after mean loss_lm tensor device: cuda:0, shape: torch.Size([])
loss_clsf calculated
loss_clsf : 0.7298979759216309
loss_clsf tensor device: cuda:0, shape: torch.Size([])
sumloss tensor device: cuda:0, shape: torch.Size([])

Why it is stuck when cuda:0 loss tensor starts to backward?
…