Thanks for flagging, the input sizes to the conv layers seem a bit unconventional so I’m wondering if that is causing a slowdown. Are these sizes part of an actual model?
cc @dskhudia
I tried printing the model
Model(
  (quant_input): Quantize(scale=tensor([1.]), zero_point=tensor([0]), dtype=torch.quint8)
  (dequant_output): DeQuantize()
  (db1): DownBlockQ(
    (quant_input): Quantize(scale=tensor([1.]), zero_point=tensor([0]), dtype=torch.quint8)
    (dequant_output): DeQuantize()
    (conv1): QuantizedConvReLU2d(22, 22, kernel_size=(4, 4), stride=(2, 2), scale=1.0, zero_point=0, padding=(1, 1), groups=22)
    (bn1): Identity()
    (relu1): Identity()
    (conv2): QuantizedConvReLU2d(22, 44, kernel_size=(1, 1), stride=(1, 1), scale=1.0, zero_point=0)
    (bn2): Identity()
    (relu2): Identity()
  )
  (db2): DownBlockQ(
    (quant_input): Quantize(scale=tensor([1.]), zero_point=tensor([0]), dtype=torch.quint8)
    (dequant_output): DeQuantize()
    (conv1): QuantizedConvReLU2d(44, 44, kernel_size=(4, 4), stride=(2, 2), scale=1.0, zero_point=0, padding=(1, 1), groups=44)
    (bn1): Identity()
    (relu1): Identity()
    (conv2): QuantizedConvReLU2d(44, 88, kernel_size=(1, 1), stride=(1, 1), scale=1.0, zero_point=0)
    (bn2): Identity()
    (relu2): Identity()
  )
  (db3): DownBlockQ(
    (quant_input): Quantize(scale=tensor([1.]), zero_point=tensor([0]), dtype=torch.quint8)
    (dequant_output): DeQuantize()
    (conv1): QuantizedConvReLU2d(88, 88, kernel_size=(4, 4), stride=(2, 2), scale=1.0, zero_point=0, padding=(1, 1), groups=88)
    (bn1): Identity()
    (relu1): Identity()
    (conv2): QuantizedConvReLU2d(88, 176, kernel_size=(1, 1), stride=(1, 1), scale=1.0, zero_point=0)
    (bn2): Identity()
    (relu2): Identity()
  )
)