It does throw that error in the isolated example that you posted above but not in the full framework. But I think I’ve identified the issue. What I’m actually doing outside of SplitDilatedCNN resembles this:
class Dummy(nn.Module):

    def __init__(self,sdcnn):
        super(Dummy, self).__init__()
        self.sdcnn = sdcnn

    def foo(self,a):
        b = self.sdcnn(a)
        a[:,:,10,10] = b[:,:,10,10]
        return a

    def forward(self,x):
        x = self.foo(x)
        return x

x = torch.randn(1, 4, 24, 24,device='cuda:0')
f = Dummy(SplitDilatedCNN()).to(device='cuda:0')
y = f(x)
y.mean().backward()

However, now x==y. What’s happening? Surely, a and b are different, so at least I would expect x[:,:,10,10] != y[:,:,10,10].