Hugging Face's logo

gokulsrinivasagan
/
bert_uncased_L-2_H-128_A-2_cola

Text Classification
Transformers
TensorBoard
Safetensors
English
bert
Generated from Trainer
Eval Results
Inference Endpoints
Model card Files Metrics Community
bert_uncased_L-2_H-128_A-2_cola / README.md
gokulsrinivasagan's picture
gokulsrinivasagan
End of training
c7ea0d3 verified
preview code
|
raw
history blame contribute delete
3.12 kB
metadata
library_name: transformers
language:
  - en
license: apache-2.0
base_model: google/bert_uncased_L-2_H-128_A-2
tags:
  - generated_from_trainer
datasets:
  - glue
metrics:
  - matthews_correlation
  - accuracy
model-index:
  - name: bert_uncased_L-2_H-128_A-2_cola
    results:
      - task:
          name: Text Classification
          type: text-classification
        dataset:
          name: GLUE COLA
          type: glue
          args: cola
        metrics:
          - name: Matthews Correlation
            type: matthews_correlation
            value: 0.00286100001416597
          - name: Accuracy
            type: accuracy
            value: 0.690316379070282

bert_uncased_L-2_H-128_A-2_cola

This model is a fine-tuned version of google/bert_uncased_L-2_H-128_A-2 on the GLUE COLA dataset. It achieves the following results on the evaluation set:

  • Loss: 0.6155
  • Matthews Correlation: 0.0029
  • Accuracy: 0.6903

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 256
  • eval_batch_size: 256
  • seed: 10
  • optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • num_epochs: 50

Training results

Training Loss Epoch Step Validation Loss Matthews Correlation Accuracy
0.6288 1.0 34 0.6191 0.0 0.6913
0.6087 2.0 68 0.6184 0.0 0.6913
0.6076 3.0 102 0.6176 0.0 0.6913
0.6066 4.0 136 0.6169 0.0 0.6913
0.605 5.0 170 0.6170 0.0 0.6913
0.6018 6.0 204 0.6164 0.0 0.6913
0.5976 7.0 238 0.6163 0.0 0.6913
0.5871 8.0 272 0.6159 0.0464 0.6922
0.5824 9.0 306 0.6155 0.0029 0.6903
0.5711 10.0 340 0.6198 0.0198 0.6702
0.5591 11.0 374 0.6221 0.0685 0.6721
0.5496 12.0 408 0.6284 0.1240 0.6702
0.5397 13.0 442 0.6350 0.1096 0.6548
0.529 14.0 476 0.6423 0.0951 0.6433

Framework versions

  • Transformers 4.46.3
  • Pytorch 2.2.1+cu118
  • Datasets 2.17.0
  • Tokenizers 0.20.3