PyTorch/Libtorch 1.4.0
ABOUT CNN:

Make a model just like MobileNetV3
Do post-training static quantization with fbgemm
The model size is reduced to a quarter of the original, the inferring speed is reduce to a half of the original, and the CPU usage is about 2400%, that means the default OMP_NUM_THREADS is 24
Do “export OMP_NUM_THREADS=1”, the inferring speed is increased to 3 times the original
Do “export OMP_NUM_THREADS=6”, the inferring speed is closed to the original

After more testing, I found that the problem is in depth-wise conv where groups is not 1.
My question is “Is this normal?”
ABOUT RNN:

Make a model with 2 LSTMs
Do post-training dynamic quantization
The model size is reduced to a quarter of the original, the inferring speed is no significantly changed

My question is “Is this normal?”