End of training

README.md

@@ -4,6 +4,8 @@ tags:
 - trl
 - sft
 - generated_from_trainer
+datasets:
+- generator
 model-index:
 - name: mistral-supervised
   results: []
@@ -14,7 +16,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 # mistral-supervised
 
-This model is a fine-tuned version of [gulmert89/mistral-pretraining](https://huggingface.co/gulmert89/mistral-pretraining) on an unknown dataset.
+This model is a fine-tuned version of [gulmert89/mistral-pretraining](https://huggingface.co/gulmert89/mistral-pretraining) on the generator dataset.
 
 ## Model description
 
@@ -39,7 +41,7 @@ The following hyperparameters were used during training:
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- num_epochs: 1
+- num_epochs: 3
 
 ### Training results
 

config.json

@@ -16,6 +16,7 @@
   "num_attention_heads": 16,
   "num_hidden_layers": 4,
   "num_key_value_heads": 8,
+  "pad_token_id": 2,
   "rms_norm_eps": 1e-06,
   "rope_theta": 10000.0,
   "sliding_window": 768,

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9b13837032c2aac57fe54a6d285133915c280a5968f9bf19adf2e1d2b3be66ce
+oid sha256:917edbb6ce453ce0076c8b23aaa79b4e71ae0440b013bcc195dd3db4414e0d6d
 size 338197712

tokenizer.json

@@ -1,11 +1,6 @@
 {
   "version": "1.0",
-  "truncation": {
-    "direction": "Right",
-    "max_length": 512,
-    "strategy": "LongestFirst",
-    "stride": 0
-  },
+  "truncation": null,
   "padding": null,
   "added_tokens": [
     {

tokenizer_config.json

@@ -38,7 +38,7 @@
   "pad_to_multiple_of": null,
   "pad_token": "</s>",
   "pad_token_type_id": 0,
-  "padding_side": "left",
+  "padding_side": "right",
   "sp_model_kwargs": {},
   "spaces_between_special_tokens": false,
   "stride": 0,

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ab74025285769d8eb265f48b8f700fac0aa10949177e29b310aee1bd4b16a382
+oid sha256:c8209e8a32ef804268b5e35de8fe38030f954cef6085aad72f9139a472a46c2c
 size 5432