zephyr-7b-ultra-p-0.01

This model is a fine-tuned version of alignment-handbook/zephyr-7b-sft-full on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5161
• Rewards/chosen: -0.5177
• Rewards/rejected: -1.9357
• Rewards/accuracies: 0.7266
• Rewards/margins: 1.4180
• Logps/rejected: -266.8364
• Logps/chosen: -235.1851
• Logits/rejected: -2.5627
• Logits/chosen: -2.6317

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-07
• train_batch_size: 1
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 8
• gradient_accumulation_steps: 8
• total_train_batch_size: 64
• total_eval_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1.0

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.5714	0.1030	100	0.5845	-0.3644	-0.9851	0.6953	0.6207	-257.3301	-233.6515	-2.5916	-2.6558
0.5551	0.2060	200	0.5465	-0.5045	-1.4338	0.7031	0.9293	-261.8167	-235.0524	-2.5073	-2.5706
0.5486	0.3090	300	0.5434	-0.4619	-1.3366	0.6719	0.8747	-260.8454	-234.6267	-2.5103	-2.5803
0.5353	0.4120	400	0.5366	-0.5676	-2.1840	0.7422	1.6165	-269.3194	-235.6834	-2.5904	-2.6560
0.5205	0.5150	500	0.5257	-0.3684	-1.8394	0.6875	1.4710	-265.8735	-233.6917	-2.6090	-2.6763
0.5187	0.6180	600	0.5268	-0.3331	-1.8290	0.7188	1.4959	-265.7695	-233.3386	-2.5558	-2.6246
0.5338	0.7210	700	0.5263	-0.3688	-1.8169	0.7109	1.4481	-265.6478	-233.6952	-2.5774	-2.6437
0.5469	0.8240	800	0.5206	-0.4773	-1.9169	0.7109	1.4396	-266.6483	-234.7807	-2.5759	-2.6446
0.4922	0.9270	900	0.5157	-0.4686	-1.8720	0.7031	1.4034	-266.1990	-234.6934	-2.5595	-2.6283

Framework versions

• Transformers 4.45.1
• Pytorch 2.4.1+cu121
• Datasets 3.0.0
• Tokenizers 0.20.0