Hi,
There are two factors that I think are contributing to your results:

You initialise a random tenor at the start of your program, this tensor is used to calculate the timing for each sample. I would recommend using real-world data if possible for better results or passing a new tensor for each inf step (see example bellow)
The performance difference torch.float32 and torch.float16 is so small that you are not able to measure the difference at the current precision level. You could try making the task more difficult for your model, a quick method could be to pass more data for the model to compute.

I would be interested to see if any PyTorch developers have anything to add to this.
import cv2
import numpy as np
from classifier import modelController
import torch
import time
import copy
mc = modelController()

tensor = torch.rand((50,3,224,112))
tensor = tensor.cuda()
tensor = tensor.half()
print(tensor.size())
# print(final_img.size())
total_time = 0
for i in range(50):
  t1 = time.time()
  get_person_cat(tensor[i].unsqueeze(0))
  t2 = time.time() - t1
  print("time taken: ", t2)
  total_time += t2
print("avg totatl time: ", total_time/50)