Hi  I believe casting to half precision on such a small tensor is overhead that is why you are not noticing any speed improvement.
I would suggest you try simple pure pytorch on your env with a bit bigger tensors and computational graph. Something like this could give you more reliable answer
import torch
import torchvision
import time

N, D_in, D_out = 64, 224, 1000
x = torch.randn(N, 3, D_in, D_in, device="cuda")
y = torch.randn(N, D_out, device="cuda")
model = torchvision.models.resnet18(pretrained=False).cuda()
optimizer = torch.optim.SGD(model.parameters(), lr=1e-3)
start_time = time.time()

for to in range(500):
    with torch.cuda.amp.autocast(enabled=False):
        y_pred = model(x)
        loss = torch.nn.functional.mse_loss(y_pred, y)

    loss.backward()
    optimizer.step()
    optimizer.zero_grad()

torch.cuda.synchronize()

compute_time = time.time() - start_time

print(compute_time)

I am currently on Colab Tesla P100 (which doesnt have native half-precision). But at least I can see that amp version is not worse (amp=True ~ 47.02 sec, amp=False ~ 49.44 sec)