@tom, You can test my quantized and fp32 models using “speed_benchmark_torch” using the flowing cmd:

./speed_benchmark_torch --model=fsmn_quant_script.pt --input_type=float --input_dim=“15,400” --iter=10000
./speed_benchmark_torch --model=fsmn_noquant_script.pt --input_type=float --input_dim=“15,400” --iter=10000

Here are my quant and noquant models
https://drive.google.com/file/d/1vz9wk8pMJfADzIuflgZIWee-PHWYLeAz/view?usp=sharing 3
https://drive.google.com/file/d/1CQ6z9ZElr4OfoDtC6455yoEwWAwUEesC/view?usp=sharing 2
It seems that the quantized model runs faster but the cpu usage is much higher than the fp32 models.