@hx89 the performance of the fused model is good

That means there’s something wrong on the quantization side, not the fusion side.  
Here’s the log of the relative norm errors -

  
      github.com
  
  
    raghavgurbaxani/experiments/blob/master/quantization_error.txt 1
basenet.slice1.3.0.weight , abs error =  0.07433768 , relative error =  0.014456971
basenet.slice1.7.0.weight , abs error =  0.102016546 , relative error =  0.012097403
basenet.slice1.10.0.weight , abs error =  0.14640729 , relative error =  0.0126273325
basenet.slice2.14.0.weight , abs error =  0.13131897 , relative error =  0.011922532
basenet.slice2.17.0.weight , abs error =  0.17593716 , relative error =  0.011550295
basenet.slice3.20.0.weight , abs error =  0.21453191 , relative error =  0.011749155
basenet.slice3.24.0.weight , abs error =  0.29290414 , relative error =  0.012482245
basenet.slice3.27.0.weight , abs error =  0.5628253 , relative error =  0.011207958
basenet.slice4.30.0.weight , abs error =  0.19498727 , relative error =  0.010849289
basenet.slice4.34.0.weight , abs error =  0.53952134 , relative error =  0.010825124
basenet.slice4.37.bias , abs error =  0.0 , relative error =  0.0
basenet.slice5.1.bias , abs error =  0.0 , relative error =  0.0
basenet.slice5.2.bias , abs error =  0.0 , relative error =  0.0
upconv1.conv.0.0.weight , abs error =  0.064128004 , relative error =  0.009871936
upconv1.conv.3.0.weight , abs error =  0.13272223 , relative error =  0.011580461
upconv2.conv.0.0.weight , abs error =  0.36004254 , relative error =  0.00813957
upconv2.conv.3.0.weight , abs error =  0.11151659 , relative error =  0.010093152
upconv3.conv.0.0.weight , abs error =  0.19112265 , relative error =  0.0072073564
upconv3.conv.3.0.weight , abs error =  0.07007974 , relative error =  0.008535749
upconv4.conv.0.0.weight , abs error =  0.0805448 , relative error =  0.0067224735


  This file has been truncated. show original

  
  
    
    
  
  


Can you suggest what to do next ? Is there any way to reduce these errors ? Apart from QAT ofcourse