Solved by Vasiliy_Kuznetsov in post #2 
              
              
                Hi @m.safari, when you run the quantization APIs it changes the state dict, because quantized layers can have different fields compared to their floating point counterparts.  Therefore, when you load a quantized checkpoint, the recommendation is to create the fp32 architecture, run the quantization …