Solved by supriyar in post #2 
              
              
                If all the operators in the model can be quantized, you can insert a quant and dequant at the beginning and end of the model. See (beta) Static Quantization with Eager Mode in PyTorch — PyTorch Tutorials 1.10.0+cu102 documentation for more details. 
If there are only specific layers that can be quan…