transformers pytorch electra text-classification generated_from_trainer dataset:amazon_polarity license:apache-2.0 model-index autotrain_compatible endpoints_compatible region:us
---
datasets:
- amazon_polarity
license: apache-2.0
metrics:
- accuracy
tags:
- generated_from_trainer
model-index:
- name: amazonPolarity_ELECTRA_5E
  results:
  - task:
      type: text-classification
      name: Text Classification
    dataset:
      name: amazon_polarity
      type: amazon_polarity
      config: amazon_polarity
      split: train
      args: amazon_polarity
    metrics:
    - type: accuracy
      value: 0.9333333333333333
      name: Accuracy
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# amazonPolarity_ELECTRA_5E

This model is a fine-tuned version of [google/electra-base-discriminator](https://huggingface.co/google/electra-base-discriminator) on the amazon_polarity dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3512
- Accuracy: 0.9333

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| 0.6705        | 0.03  | 50   | 0.5768          | 0.8867   |
| 0.4054        | 0.05  | 100  | 0.2968          | 0.8933   |
| 0.2461        | 0.08  | 150  | 0.2233          | 0.92     |
| 0.1795        | 0.11  | 200  | 0.2265          | 0.9333   |
| 0.2293        | 0.13  | 250  | 0.2329          | 0.9267   |
| 0.1541        | 0.16  | 300  | 0.2240          | 0.94     |
| 0.2006        | 0.19  | 350  | 0.2779          | 0.92     |
| 0.1826        | 0.21  | 400  | 0.2765          | 0.9133   |
| 0.1935        | 0.24  | 450  | 0.2346          | 0.9333   |
| 0.1887        | 0.27  | 500  | 0.2085          | 0.94     |
| 0.1688        | 0.29  | 550  | 0.2193          | 0.94     |
| 0.1884        | 0.32  | 600  | 0.1982          | 0.9467   |
| 0.189         | 0.35  | 650  | 0.1873          | 0.94     |
| 0.1564        | 0.37  | 700  | 0.2226          | 0.94     |
| 0.1733        | 0.4   | 750  | 0.2462          | 0.9333   |
| 0.1436        | 0.43  | 800  | 0.2328          | 0.94     |
| 0.1517        | 0.45  | 850  | 0.2128          | 0.9533   |
| 0.1922        | 0.48  | 900  | 0.1626          | 0.9467   |
| 0.1401        | 0.51  | 950  | 0.2391          | 0.94     |
| 0.1606        | 0.53  | 1000 | 0.2001          | 0.94     |
| 0.1597        | 0.56  | 1050 | 0.1788          | 0.9467   |
| 0.184         | 0.59  | 1100 | 0.1656          | 0.9467   |
| 0.1448        | 0.61  | 1150 | 0.1752          | 0.96     |
| 0.1575        | 0.64  | 1200 | 0.1878          | 0.9533   |
| 0.1836        | 0.67  | 1250 | 0.1416          | 0.9533   |
| 0.1378        | 0.69  | 1300 | 0.1866          | 0.9467   |
| 0.1901        | 0.72  | 1350 | 0.1654          | 0.9533   |
| 0.1697        | 0.75  | 1400 | 0.1720          | 0.9533   |
| 0.1624        | 0.77  | 1450 | 0.1700          | 0.9467   |
| 0.1487        | 0.8   | 1500 | 0.1786          | 0.94     |
| 0.1367        | 0.83  | 1550 | 0.1974          | 0.9267   |
| 0.1535        | 0.85  | 1600 | 0.1823          | 0.9267   |
| 0.1366        | 0.88  | 1650 | 0.1515          | 0.94     |
| 0.1505        | 0.91  | 1700 | 0.1527          | 0.94     |
| 0.1554        | 0.93  | 1750 | 0.1855          | 0.9467   |
| 0.1478        | 0.96  | 1800 | 0.1885          | 0.9333   |
| 0.1603        | 0.99  | 1850 | 0.1990          | 0.9467   |
| 0.1637        | 1.01  | 1900 | 0.1901          | 0.9467   |
| 0.1074        | 1.04  | 1950 | 0.1886          | 0.9533   |
| 0.0874        | 1.07  | 2000 | 0.2399          | 0.94     |
| 0.1245        | 1.09  | 2050 | 0.2107          | 0.9467   |
| 0.1175        | 1.12  | 2100 | 0.2226          | 0.94     |
| 0.1279        | 1.15  | 2150 | 0.2267          | 0.94     |
| 0.0947        | 1.17  | 2200 | 0.2342          | 0.94     |
| 0.0837        | 1.2   | 2250 | 0.2519          | 0.9467   |
| 0.1091        | 1.23  | 2300 | 0.2531          | 0.94     |
| 0.0867        | 1.25  | 2350 | 0.2519          | 0.94     |
| 0.0845        | 1.28  | 2400 | 0.2431          | 0.9467   |
| 0.0836        | 1.31  | 2450 | 0.1936          | 0.9533   |
| 0.1633        | 1.33  | 2500 | 0.1875          | 0.9333   |
| 0.1029        | 1.36  | 2550 | 0.2345          | 0.94     |
| 0.0755        | 1.39  | 2600 | 0.3028          | 0.94     |
| 0.1539        | 1.41  | 2650 | 0.2497          | 0.94     |
| 0.1055        | 1.44  | 2700 | 0.2002          | 0.9467   |
| 0.1234        | 1.47  | 2750 | 0.1763          | 0.9533   |
| 0.1312        | 1.49  | 2800 | 0.1998          | 0.94     |
| 0.1067        | 1.52  | 2850 | 0.1820          | 0.96     |
| 0.1092        | 1.55  | 2900 | 0.1903          | 0.9467   |
| 0.1209        | 1.57  | 2950 | 0.1912          | 0.9467   |
| 0.0627        | 1.6   | 3000 | 0.2208          | 0.9467   |
| 0.1121        | 1.63  | 3050 | 0.2607          | 0.9333   |
| 0.1106        | 1.65  | 3100 | 0.1852          | 0.9533   |
| 0.0724        | 1.68  | 3150 | 0.2122          | 0.9533   |
| 0.1247        | 1.71  | 3200 | 0.2112          | 0.9467   |
| 0.1247        | 1.73  | 3250 | 0.2021          | 0.9533   |
| 0.096         | 1.76  | 3300 | 0.2340          | 0.9467   |
| 0.1056        | 1.79  | 3350 | 0.2165          | 0.94     |
| 0.1055        | 1.81  | 3400 | 0.2563          | 0.94     |
| 0.1199        | 1.84  | 3450 | 0.2251          | 0.9467   |
| 0.0899        | 1.87  | 3500 | 0.1996          | 0.9533   |
| 0.109         | 1.89  | 3550 | 0.1924          | 0.9533   |
| 0.13          | 1.92  | 3600 | 0.1769          | 0.9467   |
| 0.1037        | 1.95  | 3650 | 0.2003          | 0.9533   |
| 0.0934        | 1.97  | 3700 | 0.2325          | 0.94     |
| 0.1254        | 2.0   | 3750 | 0.2037          | 0.9467   |
| 0.0619        | 2.03  | 3800 | 0.2252          | 0.9533   |
| 0.093         | 2.05  | 3850 | 0.2145          | 0.9533   |
| 0.0827        | 2.08  | 3900 | 0.2237          | 0.9533   |
| 0.0679        | 2.11  | 3950 | 0.2643          | 0.9467   |
| 0.076         | 2.13  | 4000 | 0.2287          | 0.9533   |
| 0.0526        | 2.16  | 4050 | 0.3210          | 0.9267   |
| 0.0354        | 2.19  | 4100 | 0.3259          | 0.9333   |
| 0.026         | 2.21  | 4150 | 0.3448          | 0.9333   |
| 0.0466        | 2.24  | 4200 | 0.3751          | 0.9333   |
| 0.043         | 2.27  | 4250 | 0.3122          | 0.9333   |
| 0.0521        | 2.29  | 4300 | 0.3155          | 0.9333   |
| 0.1018        | 2.32  | 4350 | 0.3066          | 0.94     |
| 0.0572        | 2.35  | 4400 | 0.2848          | 0.94     |
| 0.0903        | 2.37  | 4450 | 0.2289          | 0.9467   |
| 0.0718        | 2.4   | 4500 | 0.2661          | 0.9467   |
| 0.0689        | 2.43  | 4550 | 0.2544          | 0.9467   |
| 0.0829        | 2.45  | 4600 | 0.2816          | 0.9333   |
| 0.0909        | 2.48  | 4650 | 0.2244          | 0.94     |
| 0.0888        | 2.51  | 4700 | 0.2620          | 0.94     |
| 0.0998        | 2.53  | 4750 | 0.2773          | 0.94     |
| 0.0604        | 2.56  | 4800 | 0.2344          | 0.94     |
| 0.0619        | 2.59  | 4850 | 0.2551          | 0.9467   |
| 0.056         | 2.61  | 4900 | 0.2787          | 0.94     |
| 0.1037        | 2.64  | 4950 | 0.2388          | 0.9467   |
| 0.0858        | 2.67  | 5000 | 0.2213          | 0.94     |
| 0.0674        | 2.69  | 5050 | 0.2339          | 0.9467   |
| 0.0438        | 2.72  | 5100 | 0.2759          | 0.9467   |
| 0.0615        | 2.75  | 5150 | 0.2739          | 0.9467   |
| 0.064         | 2.77  | 5200 | 0.2488          | 0.9467   |
| 0.0824        | 2.8   | 5250 | 0.2590          | 0.9467   |
| 0.074         | 2.83  | 5300 | 0.2314          | 0.9467   |
| 0.1077        | 2.85  | 5350 | 0.2571          | 0.9467   |
| 0.0482        | 2.88  | 5400 | 0.2678          | 0.9467   |
| 0.0732        | 2.91  | 5450 | 0.2626          | 0.9333   |
| 0.0564        | 2.93  | 5500 | 0.2586          | 0.94     |
| 0.1019        | 2.96  | 5550 | 0.2706          | 0.9333   |
| 0.0675        | 2.99  | 5600 | 0.2568          | 0.9267   |
| 0.056         | 3.01  | 5650 | 0.2881          | 0.9333   |
| 0.0266        | 3.04  | 5700 | 0.2789          | 0.9467   |
| 0.0207        | 3.07  | 5750 | 0.2535          | 0.9467   |
| 0.0246        | 3.09  | 5800 | 0.2597          | 0.9467   |
| 0.0631        | 3.12  | 5850 | 0.2403          | 0.9533   |
| 0.0627        | 3.15  | 5900 | 0.2336          | 0.9533   |
| 0.1061        | 3.17  | 5950 | 0.2773          | 0.94     |
| 0.0257        | 3.2   | 6000 | 0.2587          | 0.9467   |
| 0.0375        | 3.23  | 6050 | 0.2560          | 0.9467   |
| 0.0404        | 3.25  | 6100 | 0.2851          | 0.94     |
| 0.0748        | 3.28  | 6150 | 0.3005          | 0.94     |
| 0.0384        | 3.31  | 6200 | 0.2442          | 0.9533   |
| 0.0426        | 3.33  | 6250 | 0.2618          | 0.9533   |
| 0.0611        | 3.36  | 6300 | 0.2710          | 0.9467   |
| 0.0282        | 3.39  | 6350 | 0.3200          | 0.94     |
| 0.0449        | 3.41  | 6400 | 0.3203          | 0.94     |
| 0.0508        | 3.44  | 6450 | 0.3197          | 0.94     |
| 0.0385        | 3.47  | 6500 | 0.3391          | 0.9333   |
| 0.0458        | 3.49  | 6550 | 0.3450          | 0.9333   |
| 0.0245        | 3.52  | 6600 | 0.3737          | 0.9333   |
| 0.0547        | 3.55  | 6650 | 0.2889          | 0.94     |
| 0.0398        | 3.57  | 6700 | 0.3751          | 0.9333   |
| 0.0497        | 3.6   | 6750 | 0.2748          | 0.9467   |
| 0.0466        | 3.63  | 6800 | 0.3438          | 0.9333   |
| 0.0241        | 3.65  | 6850 | 0.3279          | 0.9267   |
| 0.0631        | 3.68  | 6900 | 0.2921          | 0.94     |
| 0.0256        | 3.71  | 6950 | 0.3595          | 0.9267   |
| 0.0615        | 3.73  | 7000 | 0.3190          | 0.9333   |
| 0.0495        | 3.76  | 7050 | 0.3451          | 0.9267   |
| 0.0519        | 3.79  | 7100 | 0.3303          | 0.9333   |
| 0.0243        | 3.81  | 7150 | 0.3344          | 0.9333   |
| 0.0348        | 3.84  | 7200 | 0.3609          | 0.9333   |
| 0.0542        | 3.87  | 7250 | 0.2797          | 0.9333   |
| 0.0791        | 3.89  | 7300 | 0.2504          | 0.94     |
| 0.0272        | 3.92  | 7350 | 0.3165          | 0.9333   |
| 0.0701        | 3.95  | 7400 | 0.3039          | 0.9333   |
| 0.0866        | 3.97  | 7450 | 0.3233          | 0.9267   |
| 0.0461        | 4.0   | 7500 | 0.3114          | 0.9267   |
| 0.0486        | 4.03  | 7550 | 0.2995          | 0.94     |
| 0.0052        | 4.05  | 7600 | 0.3128          | 0.94     |
| 0.0312        | 4.08  | 7650 | 0.3723          | 0.9333   |
| 0.0277        | 4.11  | 7700 | 0.3158          | 0.94     |
| 0.0407        | 4.13  | 7750 | 0.3187          | 0.94     |
| 0.0224        | 4.16  | 7800 | 0.3258          | 0.9333   |
| 0.0335        | 4.19  | 7850 | 0.3539          | 0.9333   |
| 0.0425        | 4.21  | 7900 | 0.3391          | 0.9333   |
| 0.0394        | 4.24  | 7950 | 0.3470          | 0.9333   |
| 0.015         | 4.27  | 8000 | 0.3680          | 0.9333   |
| 0.0166        | 4.29  | 8050 | 0.3689          | 0.9333   |
| 0.0358        | 4.32  | 8100 | 0.3281          | 0.94     |
| 0.0152        | 4.35  | 8150 | 0.3391          | 0.9333   |
| 0.0235        | 4.37  | 8200 | 0.3506          | 0.94     |
| 0.0357        | 4.4   | 8250 | 0.3549          | 0.94     |
| 0.0153        | 4.43  | 8300 | 0.3564          | 0.94     |
| 0.0366        | 4.45  | 8350 | 0.3836          | 0.9333   |
| 0.0381        | 4.48  | 8400 | 0.3428          | 0.9333   |
| 0.0349        | 4.51  | 8450 | 0.3600          | 0.94     |
| 0.028         | 4.53  | 8500 | 0.3592          | 0.9333   |
| 0.0322        | 4.56  | 8550 | 0.3478          | 0.9333   |
| 0.0237        | 4.59  | 8600 | 0.3636          | 0.94     |
| 0.0398        | 4.61  | 8650 | 0.3433          | 0.9333   |
| 0.062         | 4.64  | 8700 | 0.3158          | 0.94     |
| 0.0148        | 4.67  | 8750 | 0.3435          | 0.9333   |
| 0.0197        | 4.69  | 8800 | 0.3394          | 0.9333   |
| 0.0594        | 4.72  | 8850 | 0.3336          | 0.9333   |
| 0.0426        | 4.75  | 8900 | 0.3351          | 0.9333   |
| 0.003         | 4.77  | 8950 | 0.3479          | 0.9333   |
| 0.0268        | 4.8   | 9000 | 0.3479          | 0.9333   |
| 0.0524        | 4.83  | 9050 | 0.3485          | 0.9333   |
| 0.0259        | 4.85  | 9100 | 0.3501          | 0.9333   |
| 0.0326        | 4.88  | 9150 | 0.3498          | 0.9333   |
| 0.0236        | 4.91  | 9200 | 0.3482          | 0.9333   |
| 0.0209        | 4.93  | 9250 | 0.3504          | 0.9333   |
| 0.0366        | 4.96  | 9300 | 0.3503          | 0.9333   |
| 0.0246        | 4.99  | 9350 | 0.3512          | 0.9333   |


### Framework versions

- Transformers 4.24.0
- Pytorch 1.13.0
- Datasets 2.6.1
- Tokenizers 0.13.1
