readme: ---
  tags:
  - optimum
  datasets:
  - banking77
  metrics:
  - accuracy
  model-index:
  - name: quantized-distilbert-banking77
    results:
    - task:
        name: Text Classification
        type: text-classification
      dataset:
        name: banking77
        type: banking77
      metrics:
      - name: Accuracy
        type: accuracy
        value: 0.9224
  ---
  
  
  # Quantized-distilbert-banking77
  
  This model is a statically quantized version of [optimum/distilbert-base-uncased-finetuned-banking77](https://huggingface.co/optimum/distilbert-base-uncased-finetuned-banking77) on the `banking77` dataset.
  
  The model was created using the [optimum-static-quantization](https://github.com/philschmid/optimum-static-quantization) notebook.
  
  It achieves the following results on the evaluation set:
  
  **Accuracy**
  
  - Vanilla model: 92.5%
  - Quantized model: 92.24%  
  
  > The quantized model achieves 99.72% accuracy of the fp32 model
  
  **Latency**
  
  Payload sequence length: 128  
  Instance type: AWS c6i.xlarge  
  
  | latency | vanilla transformers | quantized optimum model | improvement |
  |---------|----------------------|-------------------------|-------------|
  | p95     | 75.69ms              | 26.75ms                 | 2.83x       |
  | avg     | 57.52ms              | 24.86ms                 | 2.31x       |
  
  ## How to use
  
  ```python
  from optimum.onnxruntime import ORTModelForSequenceClassification
  from transformers import pipeline, AutoTokenizer
  
  model = ORTModelForSequenceClassification.from_pretrained("philschmid/quantized-distilbert-banking77")
  tokenizer = AutoTokenizer.from_pretrained("philschmid/quantized-distilbert-banking77")
  
  remote_clx = pipeline("text-classification",model=model, tokenizer=tokenizer)
  
  remote_clx("What is the exchange rate like on this app?")
  ```