IE_M2_1000steps_1e6rate_03beta_cSFTDPO

This model is a fine-tuned version of tsavage68/IE_M2_1000steps_1e7rate_SFT on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.3743
  • Rewards/chosen: -0.5009
  • Rewards/rejected: -8.2803
  • Rewards/accuracies: 0.4600
  • Rewards/margins: 7.7793
  • Logps/rejected: -68.6227
  • Logps/chosen: -43.8753
  • Logits/rejected: -2.8766
  • Logits/chosen: -2.8144

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-06
  • train_batch_size: 2
  • eval_batch_size: 1
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 4
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_steps: 100
  • training_steps: 1000

Training results

Training Loss Epoch Step Validation Loss Rewards/chosen Rewards/rejected Rewards/accuracies Rewards/margins Logps/rejected Logps/chosen Logits/rejected Logits/chosen
0.4505 0.4 50 0.3743 -0.4606 -7.3043 0.4600 6.8437 -65.3694 -43.7409 -2.8804 -2.8192
0.3812 0.8 100 0.3743 -0.5073 -7.6388 0.4600 7.1316 -66.4846 -43.8964 -2.8732 -2.8108
0.3119 1.2 150 0.3743 -0.4846 -7.8807 0.4600 7.3960 -67.2907 -43.8210 -2.8767 -2.8146
0.3639 1.6 200 0.3743 -0.4919 -7.9792 0.4600 7.4872 -67.6190 -43.8452 -2.8768 -2.8147
0.4332 2.0 250 0.3743 -0.4951 -8.0703 0.4600 7.5752 -67.9228 -43.8560 -2.8769 -2.8147
0.3986 2.4 300 0.3743 -0.4967 -8.1191 0.4600 7.6224 -68.0855 -43.8612 -2.8768 -2.8147
0.3986 2.8 350 0.3743 -0.4916 -8.1443 0.4600 7.6526 -68.1694 -43.8443 -2.8768 -2.8146
0.4505 3.2 400 0.3743 -0.4891 -8.2004 0.4600 7.7113 -68.3565 -43.8359 -2.8768 -2.8146
0.4505 3.6 450 0.3743 -0.4982 -8.2114 0.4600 7.7132 -68.3931 -43.8662 -2.8766 -2.8144
0.4332 4.0 500 0.3743 -0.4973 -8.2297 0.4600 7.7324 -68.4541 -43.8631 -2.8766 -2.8143
0.3292 4.4 550 0.3743 -0.4993 -8.2486 0.4600 7.7493 -68.5172 -43.8699 -2.8765 -2.8143
0.3639 4.8 600 0.3743 -0.5006 -8.2652 0.4600 7.7646 -68.5726 -43.8743 -2.8767 -2.8144
0.4505 5.2 650 0.3743 -0.4997 -8.2645 0.4600 7.7648 -68.5701 -43.8713 -2.8765 -2.8143
0.4505 5.6 700 0.3743 -0.5034 -8.2746 0.4600 7.7712 -68.6037 -43.8835 -2.8765 -2.8142
0.3639 6.0 750 0.3743 -0.5002 -8.2737 0.4600 7.7735 -68.6009 -43.8730 -2.8765 -2.8143
0.2426 6.4 800 0.3743 -0.4991 -8.2752 0.4600 7.7761 -68.6059 -43.8692 -2.8768 -2.8145
0.5025 6.8 850 0.3743 -0.4985 -8.2817 0.4600 7.7832 -68.6276 -43.8672 -2.8766 -2.8144
0.3119 7.2 900 0.3743 -0.5001 -8.2792 0.4600 7.7790 -68.6191 -43.8727 -2.8765 -2.8142
0.3466 7.6 950 0.3743 -0.5010 -8.2808 0.4600 7.7798 -68.6245 -43.8757 -2.8766 -2.8143
0.3812 8.0 1000 0.3743 -0.5009 -8.2803 0.4600 7.7793 -68.6227 -43.8753 -2.8766 -2.8144

Framework versions

  • Transformers 4.44.2
  • Pytorch 2.0.0+cu117
  • Datasets 3.0.0
  • Tokenizers 0.19.1
Downloads last month
2
Safetensors
Model size
7.24B params
Tensor type
F16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for tsavage68/IE_M2_1000steps_1e6rate_03beta_cSFTDPO