It’s quite unexpected to see an error in transforms.Normalize as internally the mean and std would be pushed to the same device as the incoming tensor as seen here. So even if you have a device mismatch (mean and std on CUDA while the incoming tensor would be on the CPU), it should still work:
norm = transforms.Normalize(
    mean=torch.randn(3).cuda(), std=torch.randn(3).cuda())

x = torch.randn(1, 3, 24, 24)
out = norm(x)

Are you sure this operation raises the error? Could you post the entire stacktrace, please?