Hugging Face's logo

dakwi
/
chessgpt2-medium-m

Text Generation
Transformers
Safetensors
gpt2
Generated from Trainer
text-generation-inference
Inference Endpoints
Model card Files Community
chessgpt2-medium-m / README.md
dakwi's picture
dakwi
End of training
9d03e12 verified
preview code
|
raw
history blame contribute delete
2.6 kB
metadata
library_name: transformers
license: mit
base_model: gpt2-medium
tags:
  - generated_from_trainer
model-index:
  - name: chessgpt2-medium-m
    results: []

chessgpt2-medium-m

This model is a fine-tuned version of gpt2-medium on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.9654

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 32
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 64
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.025
  • num_epochs: 3
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss
2.4607 0.1280 1000 1.8105
1.6726 0.2560 2000 1.4939
1.4686 0.3839 3000 1.3617
1.3601 0.5119 4000 1.2698
1.2875 0.6399 5000 1.2164
1.2366 0.7679 6000 1.1717
1.1992 0.8958 7000 1.1404
1.1646 1.0238 8000 1.1097
1.126 1.1518 9000 1.0934
1.1085 1.2798 10000 1.0705
1.0875 1.4077 11000 1.0541
1.0733 1.5357 12000 1.0388
1.059 1.6637 13000 1.0246
1.0451 1.7917 14000 1.0109
1.0327 1.9196 15000 1.0018
1.0118 2.0476 16000 0.9915
0.9862 2.1756 17000 0.9861
0.9806 2.3036 18000 0.9783
0.9757 2.4315 19000 0.9736
0.9713 2.5595 20000 0.9695
0.9675 2.6875 21000 0.9671
0.9659 2.8155 22000 0.9658
0.965 2.9434 23000 0.9654

Framework versions

  • Transformers 4.44.2
  • Pytorch 2.4.1+cu121
  • Datasets 3.0.1
  • Tokenizers 0.19.1