you can explicitly set the module/functional to use None to skip quantizing them in fx graph mode quantization:
for example:
qocnfig_dict = {“silu”: None}, full docs for the qconfig_dict api can be found here: pytorch/quantize_fx.py at master · pytorch/pytorch · GitHub 4