Thanks for your reply.
Yeah, I use data.to(device) for the input data in the forward().
I set the batch size as 24 and use 2 GPUs. I print label of each data in this batch: first label is on the device ‘cuda:0’, other 23 labels are in ‘cuda:1’, then I torch.cat() these labels to one tensor, it is in ‘cuda:1’ . And the output of the final layer is in ‘cuda:1’.
the error occurs at:  end_point, y = parallel_model(sample), end_point is the output of the final layer, y concatenated labels.
Traceback (most recent call last):
File “main_m.py”, line 113, in 
train(epoch, train_batch_logger, train_loader)
File “main_m.py”, line 47, in train
end_point, y = parallel_model(sample)
File “/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py”, line 489, in call
result = self.forward(*input, **kwargs)
File “/usr/local/lib/python2.7/dist-packages/torch/nn/parallel/data_parallel.py”, line 143, in forward
outputs = self.parallel_apply(replicas, inputs, kwargs)
File “/usr/local/lib/python2.7/dist-packages/torch/nn/parallel/data_parallel.py”, line 153, in parallel_apply
return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
File “/usr/local/lib/python2.7/dist-packages/torch/nn/parallel/parallel_apply.py”, line 83, in parallel_apply
raise output
RuntimeError: arguments are located on different GPUs at /pytorch/aten/src/THC/generic/THCTensorMathBlas.cu:253