Bracko/mistral-7B-v0.3-finetuned-ds5-error

results

This model is a fine-tuned version of mistralai/Mistral-7B-v0.3 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.8712

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: constant
• lr_scheduler_warmup_steps: 500
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
1.183	0.0793	500	1.0846
1.0006	0.1585	1000	1.0001
0.992	0.2378	1500	0.9630
0.9446	0.3171	2000	0.9525
0.9556	0.3963	2500	0.9416
0.9351	0.4756	3000	0.9288
0.916	0.5549	3500	0.9217
0.9015	0.6341	4000	0.9139
0.8955	0.7134	4500	0.9052
0.8714	0.7926	5000	0.9082
0.8952	0.8719	5500	0.8952
0.8947	0.9512	6000	0.8904
0.8493	1.0304	6500	0.8985
0.802	1.1097	7000	0.8899
0.7982	1.1890	7500	0.8960
0.8107	1.2682	8000	0.8935
0.8128	1.3475	8500	0.8853
0.8079	1.4268	9000	0.8915
0.7936	1.5060	9500	0.8879
0.8128	1.5853	10000	0.8794
0.8256	1.6646	10500	0.8786
0.817	1.7438	11000	0.8794
0.8137	1.8231	11500	0.8740
0.8242	1.9023	12000	0.8714
0.8128	1.9816	12500	0.8846
0.7425	2.0609	13000	0.8919
0.7186	2.1401	13500	0.8846
0.7382	2.2194	14000	0.8889
0.751	2.2987	14500	0.8784
0.7481	2.3779	15000	0.8819
0.7524	2.4572	15500	0.8770
0.7587	2.5365	16000	0.8856
0.756	2.6157	16500	0.8746
0.7698	2.6950	17000	0.8754
0.7658	2.7743	17500	0.8717
0.7662	2.8535	18000	0.8750
0.7657	2.9328	18500	0.8712

Framework versions

• PEFT 0.11.1
• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1