Have you tried this?: How do I save and load quantization model 15
i.e., prepare and convert steps before loading the state_dict
Also, I would expect conv+batchnorm+relu to be fused into QuantizedConvReLU2d but I think you are using relu6 and fusion of conv+batchnorm+relu6 isn’t currently supported.