Thanks @kfir_goldberg,
do I understand correctly that you’ve created a new model class that inherited from timm.models.efficientnet_lite0 and 1) placed your fuse_model() there 2) modified forward method to add quantization support? Can you share code of this class? I’m not sure where exactly to put quant/dequant.
Then you made:
model_.qconfig = torch.quantization.get_default_qconfig('fbgemm')
torch.quantization.prepare(model_, inplace=True)

and finally
evaluate(model_, criterion, train_loader, neval_batches=num_calibration_batches)
torch.quantization.convert(model_, inplace=True)

to calibrate the model with training set and convert, right?
I’m asking because when calibrating model (evaluate) I got:
RuntimeError: Could not run 'quantized::conv2d.new' with arguments from the 'CPU' backend. 'quantized::conv2d.new' is only available for these backends: [QuantizedCPU].

and I found here 1 that this is probably connected with the fact that QuantStub is not placed in the right place.
Have you followed this 2 tutorial? They say that when performance drops per channel quantization may be needed or Quantization-aware training.
Many thanks!