Hi @copythatpasta,  Thanks for your response.
I use the compiled version of libtorch from https://pytorch.org/get-started/locally/ 4
Also, I want to mention that I use CPU for inference. I get 5-6x speed when I try this on a simple model trained on MNIST data. But when I tried to script YOLOv3 model and use it in C++ I got the  same inference time.