I don’t think this is related to OP’s issue. But it looks like you are trying to load a saved quantized model into a non-quantized model. the state_dict to my knowledge is just a record of the parameters for different modules. Since quantized modules have different parameters than non-quantized ones, you won’t be able to load one into another. I believe that when you create the model before loading the state dict, you have to fuse/prepare/convert it, at which point you should be able to load the state dict per How do I save and load quantization model 4
If this is not helpful, can you make a separate post within the #quantization category so as not to hijack this thread? I am the oncall for quantization so I am responding to any new posts.