Doesn’t seem to be the issue, as the input is not also float16:
plain_conv = nn.Conv2d(10, 10, 3, 1, 1).cuda()
conv = SuperConv2d(10, 10, 3, 1, 1).cuda()
x = torch.randn(10, 10, 24, 24).cuda()

config = {'channel': 2}

with torch.cuda.amp.autocast():
    out = plain_conv(x)
    print(out.dtype)
    out = conv(out, config)

print(out.dtype)