Python 3.6.9
pytorch 1.2.0
I’m trying to implement a modified Conv2d (long story), so I subclassed it.  However, I wish to save some additional parameters (discrete, rarely changes) in the state_dict, so I have extra nn.Parameter(…, requires_grad=False).  Code stub:
class test(nn.Conv2d):
    def __init__(self, **kwargs):
        super(test, self).__init__(**kwargs)
        self.foo = nn.Parameter(torch.tensor(False), requires_grad=False)
    def forward(self, x):
        return F.conv2d(x, self.weight, self.bias, self.stride, self.padding, self.dilation, self.groups)

net = nn.Sequential(test(in_channels=3, out_channels=8, kernel_size=224),nn.Flatten())
net = nn.DataParallel(net)

For the forward pass, this works fine.  However, for the backwards pass, if there are >1 GPU, I get:
...
  File "/workspace/pytorch_mdk/train.py", line 86, in train
    loss.backward()
  File "/opt/conda/lib/python3.6/site-packages/torch/tensor.py", line 118, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/opt/conda/lib/python3.6/site-packages/torch/autograd/__init__.py", line 93, in backward
    allow_unreachable=True)  # allow_unreachable flag
  File "/opt/conda/lib/python3.6/site-packages/torch/autograd/function.py", line 77, in apply
    return self._forward_cls.backward(self, *args)
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/parallel/_functions.py", line 32, in backward
    return (None,) + ReduceAddCoalesced.apply(ctx.input_device, ctx.num_inputs, *grad_outputs)
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/parallel/_functions.py", line 43, in forward
    return comm.reduce_add_coalesced(grads, destination)
  File "/opt/conda/lib/python3.6/site-packages/torch/cuda/comm.py", line 121, in reduce_add_coalesced
    flat_result = reduce_add(flat_tensors, destination)
  File "/opt/conda/lib/python3.6/site-packages/torch/cuda/comm.py", line 77, in reduce_add
    nccl.reduce(inputs, outputs, root=nccl_root)
  File "/opt/conda/lib/python3.6/site-packages/torch/cuda/nccl.py", line 60, in reduce
    torch._C._nccl_reduce(inputs, outputs, root, op, streams, comms)
RuntimeError: Unconvertible NCCL type


Adding some print statements to torch/cuda/nccl.py, we can see that it’s given a bunch of tensor([False], device=‘cuda:0’), tensor([False], device=‘cuda:1’), etc.  The C code that it calls apparently does not like torch.bool, so it dumps the above RuntimeError.
Is this a bug?  Am I doing it wrong?
In the short term, I’ll probably have to change bool parameters to torch.uint8 or something.