Hi @Vasiliy_Kuznetsov, thanks for taking the time to answer and debug the model.
I have tried method 2, however there are other operations within the model that require dequant and quant and I end up modifying too much of the code.
Moreover, I tried method 3, but I still have some issues: here is a snippet of the code used for export using fx graph mode quantization, I put it in export.py right after line 61:
model_to_quantize = copy.deepcopy(model)
model_to_quantize.eval()
qconfig = torch.quantization.get_default_qconfig('qnnpack')
qconfig_dict = {"": qconfig}

from torch.quantization.quantize_fx import prepare_fx, convert_fx

prepared_model = prepare_fx(model_to_quantize, qconfig_dict)
quantized_model = convert_fx(prepared_model)
yy = quantized_model(img)

but then, when it’s being traced ts = torch.jit.trace(quantized_model, img) I get a very strange output which I find difficult to interpret. Also the predictions performed with the quantized model are not correct, I get a lot’s of small bounding boxes. It looks something like this (I won’t copy all of it here because it’s very large):
<eval_with_key_5>:7: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  quantize_per_tensor_1 = torch.quantize_per_tensor(getitem, model_0_input_scale_0, model_0_input_zero_point_0, model_0_input_dtype_0);  getitem = model_0_input_scale_0 = model_0_input_zero_point_0 = model_0_input_dtype_0 = None
<eval_with_key_5>:12: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  quantize_per_tensor_2 = torch.quantize_per_tensor(getitem_1, model_0_input_scale_1, model_0_input_zero_point_1, model_0_input_dtype_1);  getitem_1 = model_0_input_scale_1 = model_0_input_zero_point_1 = model_0_input_dtype_1 = None
...

Maybe you can help me with the source of these warnings or at least point me in the right direction. Thanks for your time!