transformers pytorch distilbert text-classification generated_from_trainer dataset:clinc_oos base_model:distilbert/distilbert-base-uncased base_model:finetune:distilbert/distilbert-base-uncased license:apache-2.0 model-index autotrain_compatible endpoints_compatible region:us
---
base_model: distilbert-base-uncased
datasets:
- clinc_oos
license: apache-2.0
metrics:
- accuracy
tags:
- generated_from_trainer
model-index:
- name: distilbert-base-uncased-distilled-clinc
  results:
  - task:
      type: text-classification
      name: Text Classification
    dataset:
      name: clinc_oos
      type: clinc_oos
      config: plus
      split: validation
      args: plus
    metrics:
    - type: accuracy
      value: 0.9458064516129032
      name: Accuracy
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# distilbert-base-uncased-distilled-clinc

This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the clinc_oos dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3215
- Accuracy: 0.9458

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 384
- eval_batch_size: 384
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 60

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| No log        | 1.0   | 40   | 3.4217          | 0.1958   |
| No log        | 2.0   | 80   | 2.9574          | 0.6029   |
| No log        | 3.0   | 120  | 2.4741          | 0.7294   |
| No log        | 4.0   | 160  | 2.0307          | 0.7926   |
| No log        | 5.0   | 200  | 1.6384          | 0.8339   |
| No log        | 6.0   | 240  | 1.3096          | 0.8658   |
| No log        | 7.0   | 280  | 1.0421          | 0.8974   |
| 2.2117        | 8.0   | 320  | 0.8340          | 0.9126   |
| 2.2117        | 9.0   | 360  | 0.6804          | 0.9216   |
| 2.2117        | 10.0  | 400  | 0.5784          | 0.93     |
| 2.2117        | 11.0  | 440  | 0.5109          | 0.9348   |
| 2.2117        | 12.0  | 480  | 0.4665          | 0.9387   |
| 2.2117        | 13.0  | 520  | 0.4352          | 0.9387   |
| 2.2117        | 14.0  | 560  | 0.4149          | 0.9406   |
| 2.2117        | 15.0  | 600  | 0.3973          | 0.9439   |
| 0.4743        | 16.0  | 640  | 0.3867          | 0.9429   |
| 0.4743        | 17.0  | 680  | 0.3793          | 0.9426   |
| 0.4743        | 18.0  | 720  | 0.3732          | 0.9435   |
| 0.4743        | 19.0  | 760  | 0.3649          | 0.9445   |
| 0.4743        | 20.0  | 800  | 0.3583          | 0.9455   |
| 0.4743        | 21.0  | 840  | 0.3577          | 0.9448   |
| 0.4743        | 22.0  | 880  | 0.3520          | 0.9432   |
| 0.4743        | 23.0  | 920  | 0.3488          | 0.9465   |
| 0.2577        | 24.0  | 960  | 0.3470          | 0.9458   |
| 0.2577        | 25.0  | 1000 | 0.3434          | 0.9471   |
| 0.2577        | 26.0  | 1040 | 0.3427          | 0.9465   |
| 0.2577        | 27.0  | 1080 | 0.3407          | 0.9452   |
| 0.2577        | 28.0  | 1120 | 0.3389          | 0.9461   |
| 0.2577        | 29.0  | 1160 | 0.3377          | 0.9465   |
| 0.2577        | 30.0  | 1200 | 0.3380          | 0.9458   |
| 0.2577        | 31.0  | 1240 | 0.3338          | 0.9461   |
| 0.219         | 32.0  | 1280 | 0.3348          | 0.9465   |
| 0.219         | 33.0  | 1320 | 0.3334          | 0.9461   |
| 0.219         | 34.0  | 1360 | 0.3314          | 0.9468   |
| 0.219         | 35.0  | 1400 | 0.3290          | 0.9481   |
| 0.219         | 36.0  | 1440 | 0.3292          | 0.9477   |
| 0.219         | 37.0  | 1480 | 0.3296          | 0.9458   |
| 0.219         | 38.0  | 1520 | 0.3290          | 0.9461   |
| 0.219         | 39.0  | 1560 | 0.3267          | 0.9458   |
| 0.2039        | 40.0  | 1600 | 0.3281          | 0.9458   |
| 0.2039        | 41.0  | 1640 | 0.3272          | 0.9458   |
| 0.2039        | 42.0  | 1680 | 0.3245          | 0.9468   |
| 0.2039        | 43.0  | 1720 | 0.3260          | 0.9461   |
| 0.2039        | 44.0  | 1760 | 0.3244          | 0.9455   |
| 0.2039        | 45.0  | 1800 | 0.3245          | 0.9458   |
| 0.2039        | 46.0  | 1840 | 0.3243          | 0.9455   |
| 0.2039        | 47.0  | 1880 | 0.3235          | 0.9455   |
| 0.1965        | 48.0  | 1920 | 0.3228          | 0.9455   |
| 0.1965        | 49.0  | 1960 | 0.3232          | 0.9465   |
| 0.1965        | 50.0  | 2000 | 0.3228          | 0.9461   |
| 0.1965        | 51.0  | 2040 | 0.3232          | 0.9468   |
| 0.1965        | 52.0  | 2080 | 0.3220          | 0.9461   |
| 0.1965        | 53.0  | 2120 | 0.3211          | 0.9465   |
| 0.1965        | 54.0  | 2160 | 0.3217          | 0.9458   |
| 0.1965        | 55.0  | 2200 | 0.3219          | 0.9465   |
| 0.1929        | 56.0  | 2240 | 0.3216          | 0.9461   |
| 0.1929        | 57.0  | 2280 | 0.3218          | 0.9465   |
| 0.1929        | 58.0  | 2320 | 0.3212          | 0.9458   |
| 0.1929        | 59.0  | 2360 | 0.3214          | 0.9455   |
| 0.1929        | 60.0  | 2400 | 0.3215          | 0.9458   |


### Framework versions

- Transformers 4.34.0
- Pytorch 2.0.1
- Datasets 2.14.5
- Tokenizers 0.14.1
