readme: ---
  pipeline_tag: text-classification
  datasets:
  - glue
  metrics:
  - accuracy
  tags:
  - roberta
  ---
  
  **task**: `text-classification`
  
  Fixed parameters:
  * **model_name_or_path**: `Bhumika/roberta-base-finetuned-sst2`
  * **dataset**:
      * **path**: `glue`
      * **eval_split**: `validation`
      * **data_keys**: `{'primary': 'sentence'}`
      * **ref_keys**: `['label']`
      * **name**: `sst2`
  * **quantization_approach**: `dynamic`
  * **node_exclusion**: `[]`
  * **per_channel**: `False`
  * **framework**: `onnxruntime`
  * **framework_args**:
      * **opset**: `15`
      * **optimization_level**: `1`
  * **aware_training**: `False`
  
  Benchmarked parameters:
  * **operators_to_quantize**: `['Add', 'MatMul']`,  `['Add']`
  
  ## Evaluation
  Below, time metrics for
  * Batch size: 8
  * Input length: 128
  | operators_to_quantize |     | latency_mean (original, ms) | latency_mean (optimized, ms) |     | throughput (original, /s) | throughput (optimized, /s) |     | accuracy (original) | accuracy (optimized) |
  | :-------------------: | :-: | :-------------------------: | :--------------------------: | :-: | :-----------------------: | :------------------------: | :-: | :-----------------: | :------------------: |
  |  `['Add', 'MatMul']`  |  \|  |           619.76            |            161.66            |  \|  |           1.80            |            6.20            |  \|  |        1.000        |        1.000         |
  |       `['Add']`       |  \|  |           611.74            |            478.48            |  \|  |           1.80            |            2.20            |  \|  |        1.000        |        1.000         |
  