Only the quantized model will work if all the layers were quantized is it right? Or else we need to dequantize the parameters again before it passes through not quantized layer is it so?

You can insert QuantStub, DequantStub blocks around the code that can be quantized. Please see https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html 21 for an example of this.

May I know that the changes in that PR is applicable for torch CPU version or not?

The change is applicable for CPU, to get these changes you can either build pytorch from source or install from pytorch nightly.