Makes sense.  The torchvision.models.quantization.inception_v3(pretrained=True, aux_logits=False, quantize=True) line is torchvision’s best effort to provide a pretrained model ready for quantization for use cases where the default fp32 pretrained weights are fine.  Unfortunately, if you need to load a different version of floating point weights, a mapping of the state dict is required.
Here is a code snippet which does this for an unrelated model, but the principle is the same:
        def get_new_bn_key(old_bn_key):
            # tries to adjust the key for conv-bn fusion, where
            # root
            #   - conv
            #   - bn
            #
            # becomes
            #
            # root
            #   - conv
            #     - bn
            return old_bn_key.replace(".bn.", ".conv.bn.")

        non_qat_to_qat_state_dict_map = {}
        for key in original_state_dict.keys():
            if key in new_state_dict.keys():
                non_qat_to_qat_state_dict_map[key] = key
            else:
                maybe_new_bn_key = get_new_bn_key(key)
                if maybe_new_bn_key in new_state_dict.keys():
                    non_qat_to_qat_state_dict_map[key] = maybe_new_bn_key
        ...
        # when loading the state dict, use the mapping created above

We are planning to release a tool soon (hopefully v1.8) to automate all of this, so this should get easier in the near future.