I guess you might be using an older torchvision release, since the operation is available via torchvision.ops.DeformConv2d and this PR 1 added autocast support ~1 year ago.
This code snippet works fine for me:
from torchvision import ops

deform_layer = ops.DeformConv2d(in_channels=3, out_channels=64, kernel_size=3).cuda()
x = torch.randn((1,3,9,9)).cuda()
offset = torch.randn(1,3*3*2,7,7).cuda()
with torch.cuda.amp.autocast():
    out = deform_layer(x, offset)