Hi @supriyar @raghuramank100,
Regarding CPU usage i debug the usage with



 supriyar:

torch.autograd.profiler


Here I’m listing the output:

Usage of Float model:

Name                      Self CPU total %  Self CPU total   CPU total %      CPU total        CPU time avg     Number of Calls

slow_conv_dilated2d       25.34%           127.183ms        98.72%           495.549ms        42.081us         11776
size                      15.79%           79.242ms         15.79%           79.242ms         0.463us          171097
_cat                      9.76%            49.007ms         11.61%           58.285ms         2.534ms          23
mkldnn_convolution        9.67%            48.534ms         19.48%           97.764ms         1.397ms          70
threshold                 6.45%            32.364ms         6.52%            32.726ms         1.091ms          30
slice                     5.26%            26.391ms         9.76%            49.012ms         2.065us          23737
native_layer_norm         3.79%            19.017ms         7.74%            38.849ms         669.817us        58
convolution              3.71%            18.632ms         86.18%           432.610ms        7.459ms          58
empty                     2.81%            14.117ms         2.82%            14.143ms         1.179us          11991
select                    2.65%            13.278ms         9.45%            47.425ms         4.027us          11778
as_strided                2.52%            12.638ms         2.52%            12.638ms         0.530us          23837
fill                     2.37%            11.903ms         2.38%            11.933ms         2.026us          5889
add                       2.23%            11.208ms         4.53%            22.760ms         421.489us        54
total time : 502 ms


Cpu usage of Quantized model without fusing:

Name                         Self CPU total %  Self CPU total   CPU total %      CPU total        CPU time avg     Number of Calls

quantized::conv1d            81.39%           454.615ms        81.39%           454.615ms        7.838ms          58
quantized::add               9.25%            51.657ms         9.25%            51.657ms         1.987ms          26
quantized::layer_norm        7.62%            42.582ms         7.62%            42.582ms         1.468ms          29
relu                         0.63%            3.513ms          1.31%            7.296ms          121.595us        60
quantized::mul               0.59%            3.322ms          0.59%            3.322ms          3.322ms          1
quantized::linear            0.18%            1.019ms          0.18%            1.019ms          1.019ms          1
total time : 558ms


CPU usage of quantized model after fusing:

Name                         Self CPU total %  Self CPU total   CPU total %      CPU total        CPU time avg     Number of Calls

quantized::conv1d            41.77%           239.086ms        41.77%           239.086ms        7.970ms          30
quantized::conv1d_relu       39.94%           228.633ms        39.94%           228.633ms        8.165ms          28
quantized::add               9.35%            53.523ms         9.35%            53.523ms         2.059ms          26
quantized::layer_norm        7.67%            43.932ms         7.67%            43.932ms         1.515ms          29
quantized::mul               0.62%            3.564ms          0.62%            3.564ms          3.564ms          1
index_add_                   0.27%            1.542ms          0.54%            3.100ms          1.550ms          2
quantized::linear            0.18%            1.017ms          0.18%            1.017ms          1.017ms          1
relu                         0.06%            370.631us        0.13%            762.027us        190.507us        4
total time : 572 ms

If you see these three usages conv1d is taking high usage after quantization

quantized::conv1d            239.086ms
quantized::conv1d_relu         228.633ms
slow_conv_dilated2d         127.183ms

Could anyone have any view why this was happened i.e., quantized conv1d increased CPU usage?
Thanks @supriyar @raghuramank100,
Aravind.