readme: ---
  license: apache-2.0
  datasets:
  - mnli
  metrics:
  - accuracy
  tags:
  - sequence-classification
  - int8
  ---
   # Quantized BERT-base MNLI model with 90% of usntructured sparsity
   The pruned and quantized model in the OpenVINO IR. The pruned model was taken from this [source](https://huggingface.co/neuralmagic/oBERT-12-downstream-pruned-unstructured-90-mnli) and quantized with the code below using HF Optimum for OpenVINO:
   
   ```python
  from functools import partial
  from transformers import AutoModelForSequenceClassification, AutoTokenizer
  from optimum.intel.openvino import OVConfig, OVQuantizer
  
  model_id = "neuralmagic/oBERT-12-downstream-pruned-unstructured-90-mnli" #"typeform/distilbert-base-uncased-mnli" 
  model = AutoModelForSequenceClassification.from_pretrained(model_id)
  tokenizer = AutoTokenizer.from_pretrained(model_id)
  save_dir = "./nm_mnli_90"
  
  def preprocess_function(examples, tokenizer):
      return tokenizer(examples["premise"], examples["hypothesis"], padding="max_length", max_length=128, truncation=True)
  
  # Load the default quantization configuration detailing the quantization we wish to apply
  quantization_config = OVConfig()
  # Instantiate our OVQuantizer using the desired configuration
  quantizer = OVQuantizer.from_pretrained(model, feature="sequence-classification")
  # Create the calibration dataset used to perform static quantization
  
  calibration_dataset = quantizer.get_calibration_dataset(
      "glue",
      dataset_config_name="mnli",
      preprocess_function=partial(preprocess_function, tokenizer=tokenizer),
      num_samples=100,
      dataset_split="train",
  )
  # Apply static quantization and export the resulting quantized model to OpenVINO IR format
  quantizer.quantize(
      quantization_config=quantization_config,
      calibration_dataset=calibration_dataset,
      save_directory=save_dir,
  )
  # Save the tokenizer
  tokenizer.save_pretrained(save_dir)
   ```