Moiz2517/Qwen2.5-Coder-3B-Python-Evaluator-bnb-4bit

results

This model is a fine-tuned version of Qwen/Qwen2.5-Coder-3B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.1720

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 8
• optimizer: Use OptimizerNames.PAGED_ADAMW_8BIT with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 2
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
1.3062	0.9849	49	1.2361
1.2408	1.9849	98	1.2071
1.2022	2.9849	147	1.1921
1.1692	3.9849	196	1.1827
1.14	4.9849	245	1.1768
1.1199	5.9849	294	1.1735
1.0947	6.9849	343	1.1710
1.0815	7.9849	392	1.1709
1.0674	8.9849	441	1.1716
1.04	9.9849	490	1.1720

Framework versions

• PEFT 0.14.0
• Transformers 4.48.3
• Pytorch 2.5.1+cu124
• Datasets 3.3.2
• Tokenizers 0.21.0