Hi, I use the Reverse Gradient as mentioned in the reply, and I work well with single GPU, but I meet some bug when i use DataParallel with gradient reverse layer.
  File "train_0.py", line 352, in run_train
    loss.backward()
  File "/home/user/miniconda3/envs/torch/lib/python3.6/site-packages/torch/autograd/variable.py", line 120, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/user/miniconda3/envs/torch/lib/python3.6/site-packages/torch/autograd/__init__.py", line 81, in backward
    variables, grad_variables, retain_graph, create_graph)
RuntimeError: arguments are located on different GPUs at /home/user/pytorch/aten/src/THC/generated/../generic/THCTensorMathPointwise.cu:231