I see. So you are not using DataLoader or anything like that but just calling that function on each image separately?
In your function
def normalizeCvImage(image_cv, device):
    image = torch.Tensor(image_cv).to(device)
    image = image.permute(2, 0, 1).unsqueeze(0)
    image = (image - 127.5) / 127.5
    return image

you have this “early” to(device) whereas for the other, you do it at the end of the pipeline. Since you are using CUDA, this could explain the difference maybe. I.e., for the second “lower CPU” usage version, you could do
def normalizeCvImage(image_cv, device):
    image = torch.Tensor(image_cv)
    image = image.permute(2, 0, 1).unsqueeze(0)
    image = (image - 127.5) / 127.5
    return image.to(device)

and see what happens