IE_M2_1000steps_1e6rate_03beta_cSFTDPO
This model is a fine-tuned version of tsavage68/IE_M2_1000steps_1e7rate_SFT on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.3743
- Rewards/chosen: -0.5009
- Rewards/rejected: -8.2803
- Rewards/accuracies: 0.4600
- Rewards/margins: 7.7793
- Logps/rejected: -68.6227
- Logps/chosen: -43.8753
- Logits/rejected: -2.8766
- Logits/chosen: -2.8144
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-06
- train_batch_size: 2
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 4
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 100
- training_steps: 1000
Training results
Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
---|---|---|---|---|---|---|---|---|---|---|---|
0.4505 | 0.4 | 50 | 0.3743 | -0.4606 | -7.3043 | 0.4600 | 6.8437 | -65.3694 | -43.7409 | -2.8804 | -2.8192 |
0.3812 | 0.8 | 100 | 0.3743 | -0.5073 | -7.6388 | 0.4600 | 7.1316 | -66.4846 | -43.8964 | -2.8732 | -2.8108 |
0.3119 | 1.2 | 150 | 0.3743 | -0.4846 | -7.8807 | 0.4600 | 7.3960 | -67.2907 | -43.8210 | -2.8767 | -2.8146 |
0.3639 | 1.6 | 200 | 0.3743 | -0.4919 | -7.9792 | 0.4600 | 7.4872 | -67.6190 | -43.8452 | -2.8768 | -2.8147 |
0.4332 | 2.0 | 250 | 0.3743 | -0.4951 | -8.0703 | 0.4600 | 7.5752 | -67.9228 | -43.8560 | -2.8769 | -2.8147 |
0.3986 | 2.4 | 300 | 0.3743 | -0.4967 | -8.1191 | 0.4600 | 7.6224 | -68.0855 | -43.8612 | -2.8768 | -2.8147 |
0.3986 | 2.8 | 350 | 0.3743 | -0.4916 | -8.1443 | 0.4600 | 7.6526 | -68.1694 | -43.8443 | -2.8768 | -2.8146 |
0.4505 | 3.2 | 400 | 0.3743 | -0.4891 | -8.2004 | 0.4600 | 7.7113 | -68.3565 | -43.8359 | -2.8768 | -2.8146 |
0.4505 | 3.6 | 450 | 0.3743 | -0.4982 | -8.2114 | 0.4600 | 7.7132 | -68.3931 | -43.8662 | -2.8766 | -2.8144 |
0.4332 | 4.0 | 500 | 0.3743 | -0.4973 | -8.2297 | 0.4600 | 7.7324 | -68.4541 | -43.8631 | -2.8766 | -2.8143 |
0.3292 | 4.4 | 550 | 0.3743 | -0.4993 | -8.2486 | 0.4600 | 7.7493 | -68.5172 | -43.8699 | -2.8765 | -2.8143 |
0.3639 | 4.8 | 600 | 0.3743 | -0.5006 | -8.2652 | 0.4600 | 7.7646 | -68.5726 | -43.8743 | -2.8767 | -2.8144 |
0.4505 | 5.2 | 650 | 0.3743 | -0.4997 | -8.2645 | 0.4600 | 7.7648 | -68.5701 | -43.8713 | -2.8765 | -2.8143 |
0.4505 | 5.6 | 700 | 0.3743 | -0.5034 | -8.2746 | 0.4600 | 7.7712 | -68.6037 | -43.8835 | -2.8765 | -2.8142 |
0.3639 | 6.0 | 750 | 0.3743 | -0.5002 | -8.2737 | 0.4600 | 7.7735 | -68.6009 | -43.8730 | -2.8765 | -2.8143 |
0.2426 | 6.4 | 800 | 0.3743 | -0.4991 | -8.2752 | 0.4600 | 7.7761 | -68.6059 | -43.8692 | -2.8768 | -2.8145 |
0.5025 | 6.8 | 850 | 0.3743 | -0.4985 | -8.2817 | 0.4600 | 7.7832 | -68.6276 | -43.8672 | -2.8766 | -2.8144 |
0.3119 | 7.2 | 900 | 0.3743 | -0.5001 | -8.2792 | 0.4600 | 7.7790 | -68.6191 | -43.8727 | -2.8765 | -2.8142 |
0.3466 | 7.6 | 950 | 0.3743 | -0.5010 | -8.2808 | 0.4600 | 7.7798 | -68.6245 | -43.8757 | -2.8766 | -2.8143 |
0.3812 | 8.0 | 1000 | 0.3743 | -0.5009 | -8.2803 | 0.4600 | 7.7793 | -68.6227 | -43.8753 | -2.8766 | -2.8144 |
Framework versions
- Transformers 4.44.2
- Pytorch 2.0.0+cu117
- Datasets 3.0.0
- Tokenizers 0.19.1
- Downloads last month
- 2
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
๐
Ask for provider support
Model tree for tsavage68/IE_M2_1000steps_1e6rate_03beta_cSFTDPO
Base model
mistralai/Mistral-7B-Instruct-v0.2
Finetuned
tsavage68/IE_M2_1000steps_1e7rate_SFT