Solved by Vasiliy_Kuznetsov in post #2 
              
              
                if an operator does not have a quantized kernel, you can run it in fp32.  For example, 
class M(torch.nn.Module):
  def __init__(...):
    ...
    self.dequant_1 = torch.quantization.DeQuantStub()
    self.quant_1 = torch.quantization.QuantStub()
    ...

  def forward(self, ...):
    ...
    # (com…