I also test the runing time. My quantized model is much slower than the fp32 model.
If I run each (quantized and non-quantized) model separately, the cpu utilization is just like the forgoing picture.
I have 4 cores. For this simple model, I think it’s enough.