I post my experience on https://github.com/pytorch/pytorch/issues/20156 27. seems like the CUDA version libtorch have sth wrong with cpu?
I test code in this tutorial: https://pytorch.org/tutorials/advanced/cpp_export.html , on my ubuntu18.04 with 6 hard cpu cores and a RTX2080.  it cost 80s to finished 1000 model.forward in c++ code, but just 25s in python code!  I can’t call at::init_num_threads, because it complains: error: 'init_num_threads' is not a member of 'at'. what’s the most strange I think is when I run c++ code, cpu is 1000%high, but when I run python code, cpu is 600% high!
I searched for a long time, but it doesn’t work.
But when I did as the tutorial: https://pytorch.org/cppdocs/installing.html 16, download libtorch from https://download.pytorch.org/libtorch/nightly/cpu/libtorch-shared-with-deps-latest.zip, it works!!! the cpu is 1200% when I run c++ code, same fast as python code: 24s!
At first, I used this download from pytorch download page:
https://download.pytorch.org/libtorch/cu101/libtorch-cxx11-abi-shared-with-deps-1.4.0.zip
So I think maybe this download just work for CUDA? And I test model.forward on CUDA in cpp, and in python, everything goes as expected: cpu is 100%, and every fast: 2.4s for 1000 forwards both in c++ and python, 10 times faster than the cpu version.
So I think, the problem is why the CUDA version libtorch is “so strange” on cpu? By “strange” I mean, cpu is 1000% high(not 600%, not 1200%) and 3 times slower than python version.