Traceback (most recent call last):
  File "main.py", line 201, in <module>
    loss_list, lr_epoch, mu_epoch = train(epoch)
  File "main.py", line 132, in train
    outputs = net(inputs)
  File "/home/lala/miniconda2/lib/python2.7/site-packages/torch/nn/modules/module.py", line 224, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/lala/miniconda2/lib/python2.7/site-packages/torch/nn/parallel/data_parallel.py", line 59, in forward
    replicas = self.replicate(self.module, self.device_ids[:len(inputs)])
  File "/home/lala/miniconda2/lib/python2.7/site-packages/torch/nn/parallel/data_parallel.py", line 64, in replicate
    return replicate(module, device_ids)
  File "/home/lala/miniconda2/lib/python2.7/site-packages/torch/nn/parallel/replicate.py", line 12, in replicate
    param_copies = Broadcast(devices)(*params)
  File "/home/lala/miniconda2/lib/python2.7/site-packages/torch/nn/parallel/_functions.py", line 19, in forward
    outputs = comm.broadcast_coalesced(inputs, self.target_gpus)
  File "/home/lala/miniconda2/lib/python2.7/site-packages/torch/cuda/comm.py", line 54, in broadcast_coalesced
    results = broadcast(_flatten_tensors(chunk), devices)
  File "/home/lala/miniconda2/lib/python2.7/site-packages/torch/cuda/comm.py", line 24, in broadcast
    nccl.broadcast(tensors)
  File "/home/lala/miniconda2/lib/python2.7/site-packages/torch/cuda/nccl.py", line 190, in broadcast
    data_type, root, comm[i], cudaStream()))
  File "/home/lala/miniconda2/lib/python2.7/site-packages/torch/cuda/nccl.py", line 118, in check_error
    raise NcclError(status)
torch.cuda.nccl.NcclError: Unhandled Cuda Error (1)

When I run without GPU, the code is fine. On v0.1.12 it is fine on GPU and CPU.
Lines with issues I believe
if use_cuda:
    net.cuda()
    net = torch.nn.DataParallel(net, device_ids=range(torch.cuda.device_count()))
    cudnn.benchmark = True