A10-Llasa-1B

This model is a fine-tuned version of HKUSTAudio/Llasa-1B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 7.4506

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• distributed_type: multi-GPU
• num_devices: 2
• total_train_batch_size: 2
• total_eval_batch_size: 2
• optimizer: Adam with betas=(0.9,0.95) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.03
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0	0	7.8569
7.1218	0.0751	1000	7.5281
7.2827	0.1503	2000	7.4962
7.2238	0.2254	3000	7.4882
7.3725	0.3006	4000	7.4697
7.2284	0.3757	5000	7.4629
6.9178	0.4509	6000	7.4605
7.2347	0.5260	7000	7.4566
7.0272	0.6012	8000	7.4574
7.0231	0.6763	9000	7.4564
7.1072	0.7515	10000	7.4533
7.0091	0.8266	11000	7.4523
7.1567	0.9018	12000	7.4519
7.0096	0.9769	13000	7.4506

Framework versions

• PEFT 0.15.2
• Transformers 4.45.2
• Pytorch 2.4.1+cu121
• Datasets 3.0.1
• Tokenizers 0.20.1