I’ve got a simple model consisting only of convolutions (even no activation between) and I wanted to benchmark it in Caffe2 on my target Android device. I followed https://pytorch.org/tutorials/advanced/super_resolution_with_caffe2.html 3
but when I run
./speed_benchmark --init_net=model_for_inference-simplified-init-net.pb --net=model_for_inference-simplified-predict-net.pb --iter=1
it runs on single thread so time is:
milliseconds per iter: 16674.4
compared to 2438.7 in TF which is able to use 8 cores
Speed benchmark was built using:
scripts/build_android.sh  -DANDROID_ABI=arm64-v8a -DANDROID_TOOLCHAIN=clang -DBUILD_BINARY=ON
On x86 setting OMP_NUM_THREADS=8 helps but not on ARM