In another post they say that pytorch is optimized for gpu only. I train the model with pytorch and copy the weights to caffe manually, which is quite optimized for inference in cpu, and moreover, you can use Intel caffe, much more optimized for Intel processor. You can try with onnx https://github.com/onnx 173 to convert your model into other frameworks that are optimized for cpu.