I think I am missing something here,
bn = nn.BatchNorm2d(3, affine=True)
x = torch.randn(1, 3, 7, 7,dtype=torch.double, requires_grad=True)
torch.autograd.gradcheck(bn, x)

I get:
RuntimeError: expected scalar type Double but found Float