Hugging Face's logo

sandernotenbaert
/
OKAI-midi-gen-v-004

Other
Transformers
TensorBoard
Safetensors
mistral
text-generation
Generated from Trainer
text-generation-inference
Model card Files
xet
Metrics Community
OKAI-midi-gen-v-004 / README.md
sandernotenbaert's picture
sandernotenbaert
Update metadata with huggingface_hub
6fc706e verified
preview code
|
raw
history blame contribute delete
2.99 kB
metadata
library_name: transformers
tags:
  - generated_from_trainer
metrics:
  - accuracy
training_config:
  vocab_size: 30000
  hidden_size: 256
  intermediate_size: 1024
  num_hidden_layers: 6
  num_attention_heads: 4
  num_key_value_heads: 4
  sliding_window: 256
  max_position_embeddings: 4096
  pad_token_id: 0
  bos_token_id: 1
  eos_token_id: 2
pipeline_tag: other
model-index:
  - name: OKAI-midi-gen-v-004
    results: []

OKAI-midi-gen-v-004

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 10.2911
  • Accuracy: 0.0003

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 8
  • eval_batch_size: 16
  • seed: 444
  • gradient_accumulation_steps: 3
  • total_train_batch_size: 24
  • optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: cosine_with_restarts
  • lr_scheduler_warmup_ratio: 0.3
  • training_steps: 2000

Training results

Training Loss Epoch Step Validation Loss Accuracy
10.2092 3.2283 100 10.2764 0.0004
9.7373 6.4565 200 10.0773 0.0020
9.2158 9.6848 300 9.9404 0.0023
8.8101 12.9130 400 9.9445 0.0027
8.3439 16.1304 500 9.9546 0.0016
7.7001 19.3587 600 9.9316 0.0012
6.9652 22.5870 700 9.9396 0.0007
6.2067 25.8152 800 9.9274 0.0007
5.5185 29.0326 900 9.9435 0.0007
4.8318 32.2609 1000 9.9918 0.0004
4.343 35.4891 1100 10.0255 0.0004
3.9477 38.7174 1200 10.0792 0.0004
3.5394 41.9457 1300 10.1247 0.0002
3.2964 45.1630 1400 10.1824 0.0003
3.0237 48.3913 1500 10.2247 0.0003
2.8621 51.6196 1600 10.2556 0.0003
2.765 54.8478 1700 10.2756 0.0001
2.7383 58.0652 1800 10.2867 0.0003
2.7324 61.2935 1900 10.2906 0.0003
2.6918 64.5217 2000 10.2911 0.0003

Framework versions

  • Transformers 4.52.3
  • Pytorch 2.6.0
  • Datasets 3.6.0
  • Tokenizers 0.21.1