Solved by supriyar in post #5 
              
              
                Only the quantized model will work if all the layers were quantized is it right? Or else we need to dequantize the parameters again before it passes through not quantized layer is it so? 

You can insert QuantStub, DequantStub blocks around the code that can be quantized. Please see https://pytorch…