Thanks for the repository.
I’ve installed a new Ubuntu 20.04 OS with the latest CUDA driver on a P6000 node and am unfortunately unable to run the code.
First issue:
Traceback (most recent call last):
  File "debug.py", line 58, in <module>
    model(inputs)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/tmp/pytorch_p6000/VnBSS/models/llcp_net.py", line 191, in forward
    x = sp_mix.permute(0, 3, 1, 2)
RuntimeError: number of dims don't match in permute

I’ve unsqueezed an additional dimension for sp_mix and spm to avoid this issue, which was followed by:
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 446, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 442, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Given groups=1, weight of size [96, 2, 1, 7], expected input[2, 1, 256, 256] to have 2 channels, but got 1 channels instead

so clearly a simple unsqueeze isn’t working. I’ve thus used sp_mix = sp_mix.unsqueeze(3).repeat(1, 1, 1, 2) for both tensors to make sure that 2 channels are used.
Afterwards, the code fails with:
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 446, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 442, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Input type (CUDAComplexFloatType) and weight type (torch.cuda.FloatTensor) should be the same

so I’m unsure how it’s executable on your other devices.