OKAI-midi-gen-v-003

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 10.9243
• Accuracy: 0.0006

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 16
• seed: 444
• gradient_accumulation_steps: 3
• total_train_batch_size: 24
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine_with_restarts
• lr_scheduler_warmup_ratio: 0.3
• training_steps: 2000

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
10.0807	3.2283	100	10.3767	0.0017
8.5059	6.4565	200	10.0807	0.0051
6.9441	9.6848	300	10.1639	0.0067
4.8834	12.9130	400	10.2550	0.0040
3.1164	16.1304	500	10.3465	0.0022
1.6129	19.3587	600	10.4677	0.0012
0.6993	22.5870	700	10.5827	0.0009
0.3373	25.8152	800	10.6865	0.0008
0.192	29.0326	900	10.7662	0.0006
0.1231	32.2609	1000	10.8070	0.0006
0.0955	35.4891	1100	10.8383	0.0006
0.0796	38.7174	1200	10.8623	0.0007
0.0714	41.9457	1300	10.8789	0.0005
0.0626	45.1630	1400	10.9002	0.0007
0.0564	48.3913	1500	10.9080	0.0007
0.0508	51.6196	1600	10.9148	0.0007
0.0494	54.8478	1700	10.9207	0.0006
0.0487	58.0652	1800	10.9227	0.0006
0.0477	61.2935	1900	10.9239	0.0006
0.0471	64.5217	2000	10.9243	0.0006

Framework versions

• Transformers 4.52.3
• Pytorch 2.6.0
• Datasets 3.6.0
• Tokenizers 0.21.1