@Vasiliy_Kuznetsov
Thanks. Apart from the layernorm problem, I found another problem. Even my network is very simple, for example, just one Linear layer without LayerNorm, the cpu usage is very high after quantization. More details can be found in in post.

  
    
    
    Quantized model has higer cpu usage on Android Mobile
  
  
    Hi all, 
Recently, I am deploying my model to Android. I found that my quantized model has very high cpu usage, even much higher than fp32 float model. So I tried to run a model with only one Linear layer like: 

Class Linear（torch.nn.Module): 

def _init_(self, idim, odim): 

self.quant= torch.quantization.QuantStub() 
self.dequant = torch.quantization.DequantStub() 
self.linear = torch.nn.Sequential(torch.nn.Linear(idim, 60), torch.nn.Linear(60, odim), torch.nn.ReLU()) 





def forward(self, …
  


This problem has been confused me for a long time.