You can control the layers to quantize by specifying quant/dequant stubs around the layers. For more details you can refer to the tutorial in (beta) Static Quantization with Eager Mode in PyTorch — PyTorch Tutorials 1.9.0+cu102 documentation 2