@jerryzh168 @Vasiliy_Kuznetsov
Irrespective of the above error, I still don’t understand how the inference time is almost the same despite the proper modules are shown to be quantized when it is printed
` #Nomal model
model_inference 5.31% 24.628ms 99.99% 463.702ms 463.702ms
#Static quantized model
model_inference 4.93% 22.530ms 99.99% 456.504ms 456.504ms
#ROI quantized model (dynamically quantizing the Linear layers)
model_inference 5.31% 24.628ms 99.99% 461.05ms 461.05ms