Hi, I have an issue in deploying yolov3 model to work fast in c++. I am working on this implementation: https://github.com/eriklindernoren/PyTorch-YOLOv3 10
I used ‘torch.jit.trace’ to save the model as following:
    example_img = torch.rand(1,3,416,416)
    with torch.jit.optimized_execution(True):
    	traced_script_module = torch.jit.trace(model, example_img)# save the converted model
    	traced_script_module.save("yolov3.pt")

I loaded the file in C++ code using torch::jit::load as following:
  torch::jit::script::Module module;
  module = torch::jit::load("/home/bahey/dev_ws/src/testorch/yolov3.pt");
  torch::NoGradGuard no_grad_guard;
  at::init_num_threads();


I calculated inference time in python and c++. I got the same inference time in both. I was expecting that C++ should be faster. My question is how can I make the model’s inference time faster in C++?
#Environment:
PyTorch Version 1.5
OS (e.g., Linux): Ubuntu 18.04
Libtorch version: Nightly cxx11 ABI
Python version:3.6