have you considered using nn.dataparallel 11?
anyway by calling .device you can retrieve the information. Example:
import torch
a=torch.zeros((10,10).cuda()
a.device.index #gpu index
a.device.type # if it is in gpu or cpu