@dhpollack @fmassa  Thank you so much. I just thought that it would be more efficient to leave the “Variable” procedure to multiple processes. It would not be necessary if it takes almost no time.
And I start to have another question now. Does it make sense to copy the data to the gpu at the data loading step?
class ToTensor(object):
    """Convert ndarrays in sample to Tensors."""

    def __init__(self, phase_cuda=False):
        self.phase_cuda = phase_cuda

    def __call__(self, sample):
        image, landmarks = sample['image'], sample['landmarks']

        # swap color axis because
        # numpy image: H x W x C
        # torch image: C X H X W
        image = image.transpose((2, 0, 1))
        return {'image': torch.from_numpy(image).cuda() if self.phase_cuda else torch.from_numpy(image),
                'landmarks': torch.from_numpy(landmarks).cuda() if self.phase_cuda else torch.from_numpy(landmarks)}