if an operator does not have a quantized kernel, you can run it in fp32.  For example,
class M(torch.nn.Module):
  def __init__(...):
    ...
    self.dequant_1 = torch.quantization.DeQuantStub()
    self.quant_1 = torch.quantization.QuantStub()
    ...

  def forward(self, ...):
    ...
    # (computation in int8)
    ...
    # convert from int8 to fp32
    x_1 = self.dequant_1(x_0)
    # run in fp32
    x_2 = self.maxpool(x_1)
    # convert back to int8, if needed
    x_3 = self.quant_q(x_2)
    ...