Training in progress, step 150, checkpoint

last-checkpoint/README.md

@@ -0,0 +1,202 @@
+---
+base_model: NousResearch/Yarn-Mistral-7b-64k
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

last-checkpoint/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "NousResearch/Yarn-Mistral-7b-64k",
+  "bias": "none",
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_dropout": 0.3,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "o_proj",
+    "up_proj",
+    "gate_proj",
+    "down_proj",
+    "k_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

last-checkpoint/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c18bf88341c296a1d7e51adac0903de4e97fb477d200e0a5bd50abfbf73fcfc8
+size 671149168

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:847113f32edd9da23b599904c977f60e91a7922918f408c419e950f009ff78ac
+size 341314196

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac41ef83a68e0cabc76856233c38710e5cc902bf4bdc040a9db4570c041a4b58
+size 14244

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:08e855e1a75677aa30cea5076b6f59cc5563c1a125648999a47f5b0b8ab9967f
+size 1064

last-checkpoint/special_tokens_map.json

@@ -0,0 +1,35 @@
+{
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

last-checkpoint/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
+size 493443

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,48 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "add_prefix_space": null,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": "<s>",
+  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "legacy": true,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "</s>",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": true
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,1108 @@
+{
+  "best_metric": 0.39922699332237244,
+  "best_model_checkpoint": "miner_id_24/checkpoint-150",
+  "epoch": 0.008191461766352206,
+  "eval_steps": 150,
+  "global_step": 150,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 5.46097451090147e-05,
+      "grad_norm": 10.157268524169922,
+      "learning_rate": 2e-05,
+      "loss": 2.6189,
+      "step": 1
+    },
+    {
+      "epoch": 5.46097451090147e-05,
+      "eval_loss": 0.7597478628158569,
+      "eval_runtime": 1190.5229,
+      "eval_samples_per_second": 4.2,
+      "eval_steps_per_second": 2.1,
+      "step": 1
+    },
+    {
+      "epoch": 0.0001092194902180294,
+      "grad_norm": 10.577338218688965,
+      "learning_rate": 4e-05,
+      "loss": 3.1498,
+      "step": 2
+    },
+    {
+      "epoch": 0.0001638292353270441,
+      "grad_norm": 13.364782333374023,
+      "learning_rate": 6e-05,
+      "loss": 3.3616,
+      "step": 3
+    },
+    {
+      "epoch": 0.0002184389804360588,
+      "grad_norm": 9.990060806274414,
+      "learning_rate": 8e-05,
+      "loss": 2.4955,
+      "step": 4
+    },
+    {
+      "epoch": 0.00027304872554507353,
+      "grad_norm": 12.803841590881348,
+      "learning_rate": 0.0001,
+      "loss": 2.6618,
+      "step": 5
+    },
+    {
+      "epoch": 0.0003276584706540882,
+      "grad_norm": 9.375833511352539,
+      "learning_rate": 0.00012,
+      "loss": 2.1728,
+      "step": 6
+    },
+    {
+      "epoch": 0.0003822682157631029,
+      "grad_norm": 9.238130569458008,
+      "learning_rate": 0.00014,
+      "loss": 2.399,
+      "step": 7
+    },
+    {
+      "epoch": 0.0004368779608721176,
+      "grad_norm": 8.292777061462402,
+      "learning_rate": 0.00016,
+      "loss": 2.0679,
+      "step": 8
+    },
+    {
+      "epoch": 0.0004914877059811324,
+      "grad_norm": 8.28408432006836,
+      "learning_rate": 0.00018,
+      "loss": 2.2563,
+      "step": 9
+    },
+    {
+      "epoch": 0.0005460974510901471,
+      "grad_norm": 7.245044231414795,
+      "learning_rate": 0.0002,
+      "loss": 2.2377,
+      "step": 10
+    },
+    {
+      "epoch": 0.0006007071961991618,
+      "grad_norm": 8.960830688476562,
+      "learning_rate": 0.0001999999286015924,
+      "loss": 1.8197,
+      "step": 11
+    },
+    {
+      "epoch": 0.0006553169413081765,
+      "grad_norm": 6.913742542266846,
+      "learning_rate": 0.00019999971440647153,
+      "loss": 2.0073,
+      "step": 12
+    },
+    {
+      "epoch": 0.0007099266864171911,
+      "grad_norm": 15.250874519348145,
+      "learning_rate": 0.00019999935741494328,
+      "loss": 2.2004,
+      "step": 13
+    },
+    {
+      "epoch": 0.0007645364315262058,
+      "grad_norm": 8.188089370727539,
+      "learning_rate": 0.00019999885762751746,
+      "loss": 1.8803,
+      "step": 14
+    },
+    {
+      "epoch": 0.0008191461766352205,
+      "grad_norm": 7.808596134185791,
+      "learning_rate": 0.00019999821504490764,
+      "loss": 2.3095,
+      "step": 15
+    },
+    {
+      "epoch": 0.0008737559217442352,
+      "grad_norm": 6.223012924194336,
+      "learning_rate": 0.00019999742966803148,
+      "loss": 1.7667,
+      "step": 16
+    },
+    {
+      "epoch": 0.0009283656668532499,
+      "grad_norm": 13.470848083496094,
+      "learning_rate": 0.0001999965014980105,
+      "loss": 1.9785,
+      "step": 17
+    },
+    {
+      "epoch": 0.0009829754119622647,
+      "grad_norm": 8.731101989746094,
+      "learning_rate": 0.00019999543053617001,
+      "loss": 1.8728,
+      "step": 18
+    },
+    {
+      "epoch": 0.0010375851570712793,
+      "grad_norm": 6.7653303146362305,
+      "learning_rate": 0.0001999942167840394,
+      "loss": 1.8614,
+      "step": 19
+    },
+    {
+      "epoch": 0.0010921949021802941,
+      "grad_norm": 8.778120040893555,
+      "learning_rate": 0.0001999928602433518,
+      "loss": 2.0676,
+      "step": 20
+    },
+    {
+      "epoch": 0.0011468046472893087,
+      "grad_norm": 7.683137893676758,
+      "learning_rate": 0.00019999136091604434,
+      "loss": 1.954,
+      "step": 21
+    },
+    {
+      "epoch": 0.0012014143923983235,
+      "grad_norm": 7.66323709487915,
+      "learning_rate": 0.00019998971880425803,
+      "loss": 2.1378,
+      "step": 22
+    },
+    {
+      "epoch": 0.001256024137507338,
+      "grad_norm": 13.016460418701172,
+      "learning_rate": 0.00019998793391033773,
+      "loss": 1.8264,
+      "step": 23
+    },
+    {
+      "epoch": 0.001310633882616353,
+      "grad_norm": 7.348866939544678,
+      "learning_rate": 0.00019998600623683218,
+      "loss": 2.0045,
+      "step": 24
+    },
+    {
+      "epoch": 0.0013652436277253675,
+      "grad_norm": 7.548125743865967,
+      "learning_rate": 0.0001999839357864941,
+      "loss": 2.1639,
+      "step": 25
+    },
+    {
+      "epoch": 0.0014198533728343823,
+      "grad_norm": 6.877038478851318,
+      "learning_rate": 0.00019998172256228,
+      "loss": 2.2839,
+      "step": 26
+    },
+    {
+      "epoch": 0.001474463117943397,
+      "grad_norm": 8.00535774230957,
+      "learning_rate": 0.0001999793665673503,
+      "loss": 2.0861,
+      "step": 27
+    },
+    {
+      "epoch": 0.0015290728630524117,
+      "grad_norm": 9.622394561767578,
+      "learning_rate": 0.00019997686780506924,
+      "loss": 2.2637,
+      "step": 28
+    },
+    {
+      "epoch": 0.0015836826081614265,
+      "grad_norm": 18.274282455444336,
+      "learning_rate": 0.000199974226279005,
+      "loss": 1.8328,
+      "step": 29
+    },
+    {
+      "epoch": 0.001638292353270441,
+      "grad_norm": 7.911862850189209,
+      "learning_rate": 0.00019997144199292968,
+      "loss": 2.0341,
+      "step": 30
+    },
+    {
+      "epoch": 0.0016929020983794559,
+      "grad_norm": 8.053264617919922,
+      "learning_rate": 0.000199968514950819,
+      "loss": 2.1631,
+      "step": 31
+    },
+    {
+      "epoch": 0.0017475118434884705,
+      "grad_norm": 7.35968017578125,
+      "learning_rate": 0.00019996544515685281,
+      "loss": 2.2252,
+      "step": 32
+    },
+    {
+      "epoch": 0.0018021215885974853,
+      "grad_norm": 5.659560203552246,
+      "learning_rate": 0.0001999622326154146,
+      "loss": 1.8681,
+      "step": 33
+    },
+    {
+      "epoch": 0.0018567313337064999,
+      "grad_norm": 6.467886924743652,
+      "learning_rate": 0.0001999588773310918,
+      "loss": 2.2596,
+      "step": 34
+    },
+    {
+      "epoch": 0.0019113410788155147,
+      "grad_norm": 5.230185508728027,
+      "learning_rate": 0.00019995537930867565,
+      "loss": 1.9119,
+      "step": 35
+    },
+    {
+      "epoch": 0.0019659508239245295,
+      "grad_norm": 6.366949558258057,
+      "learning_rate": 0.00019995173855316125,
+      "loss": 1.78,
+      "step": 36
+    },
+    {
+      "epoch": 0.002020560569033544,
+      "grad_norm": 4.931448459625244,
+      "learning_rate": 0.00019994795506974745,
+      "loss": 1.5861,
+      "step": 37
+    },
+    {
+      "epoch": 0.0020751703141425586,
+      "grad_norm": 5.934323310852051,
+      "learning_rate": 0.0001999440288638369,
+      "loss": 1.4948,
+      "step": 38
+    },
+    {
+      "epoch": 0.0021297800592515737,
+      "grad_norm": 7.572028160095215,
+      "learning_rate": 0.00019993995994103616,
+      "loss": 1.8185,
+      "step": 39
+    },
+    {
+      "epoch": 0.0021843898043605882,
+      "grad_norm": 7.979534149169922,
+      "learning_rate": 0.00019993574830715551,
+      "loss": 2.1558,
+      "step": 40
+    },
+    {
+      "epoch": 0.002238999549469603,
+      "grad_norm": 7.424667835235596,
+      "learning_rate": 0.00019993139396820903,
+      "loss": 1.809,
+      "step": 41
+    },
+    {
+      "epoch": 0.0022936092945786174,
+      "grad_norm": 6.131507873535156,
+      "learning_rate": 0.00019992689693041458,
+      "loss": 1.7222,
+      "step": 42
+    },
+    {
+      "epoch": 0.0023482190396876324,
+      "grad_norm": 6.393542289733887,
+      "learning_rate": 0.00019992225720019376,
+      "loss": 1.6676,
+      "step": 43
+    },
+    {
+      "epoch": 0.002402828784796647,
+      "grad_norm": 5.539822578430176,
+      "learning_rate": 0.00019991747478417194,
+      "loss": 1.658,
+      "step": 44
+    },
+    {
+      "epoch": 0.0024574385299056616,
+      "grad_norm": 6.343573570251465,
+      "learning_rate": 0.00019991254968917835,
+      "loss": 1.6642,
+      "step": 45
+    },
+    {
+      "epoch": 0.002512048275014676,
+      "grad_norm": 6.394305229187012,
+      "learning_rate": 0.00019990748192224577,
+      "loss": 1.8305,
+      "step": 46
+    },
+    {
+      "epoch": 0.002566658020123691,
+      "grad_norm": 8.003130912780762,
+      "learning_rate": 0.00019990227149061087,
+      "loss": 1.9036,
+      "step": 47
+    },
+    {
+      "epoch": 0.002621267765232706,
+      "grad_norm": 7.291452407836914,
+      "learning_rate": 0.00019989691840171397,
+      "loss": 2.0083,
+      "step": 48
+    },
+    {
+      "epoch": 0.0026758775103417204,
+      "grad_norm": 7.289302825927734,
+      "learning_rate": 0.00019989142266319906,
+      "loss": 2.0038,
+      "step": 49
+    },
+    {
+      "epoch": 0.002730487255450735,
+      "grad_norm": 6.551719665527344,
+      "learning_rate": 0.00019988578428291396,
+      "loss": 1.6449,
+      "step": 50
+    },
+    {
+      "epoch": 0.00278509700055975,
+      "grad_norm": 7.771378993988037,
+      "learning_rate": 0.00019988000326891006,
+      "loss": 2.0701,
+      "step": 51
+    },
+    {
+      "epoch": 0.0028397067456687646,
+      "grad_norm": 5.325085639953613,
+      "learning_rate": 0.00019987407962944245,
+      "loss": 1.5095,
+      "step": 52
+    },
+    {
+      "epoch": 0.002894316490777779,
+      "grad_norm": 7.79744815826416,
+      "learning_rate": 0.00019986801337296993,
+      "loss": 1.9184,
+      "step": 53
+    },
+    {
+      "epoch": 0.002948926235886794,
+      "grad_norm": 5.620527744293213,
+      "learning_rate": 0.00019986180450815485,
+      "loss": 1.72,
+      "step": 54
+    },
+    {
+      "epoch": 0.0030035359809958088,
+      "grad_norm": 11.069401741027832,
+      "learning_rate": 0.00019985545304386336,
+      "loss": 1.611,
+      "step": 55
+    },
+    {
+      "epoch": 0.0030581457261048234,
+      "grad_norm": 6.716678142547607,
+      "learning_rate": 0.0001998489589891651,
+      "loss": 1.9795,
+      "step": 56
+    },
+    {
+      "epoch": 0.003112755471213838,
+      "grad_norm": 6.914350509643555,
+      "learning_rate": 0.00019984232235333339,
+      "loss": 1.8914,
+      "step": 57
+    },
+    {
+      "epoch": 0.003167365216322853,
+      "grad_norm": 6.589979648590088,
+      "learning_rate": 0.00019983554314584513,
+      "loss": 1.9666,
+      "step": 58
+    },
+    {
+      "epoch": 0.0032219749614318676,
+      "grad_norm": 6.596893787384033,
+      "learning_rate": 0.00019982862137638084,
+      "loss": 1.9721,
+      "step": 59
+    },
+    {
+      "epoch": 0.003276584706540882,
+      "grad_norm": 6.24562406539917,
+      "learning_rate": 0.0001998215570548245,
+      "loss": 1.6719,
+      "step": 60
+    },
+    {
+      "epoch": 0.0033311944516498967,
+      "grad_norm": 7.659477233886719,
+      "learning_rate": 0.00019981435019126385,
+      "loss": 1.8265,
+      "step": 61
+    },
+    {
+      "epoch": 0.0033858041967589118,
+      "grad_norm": 7.833114147186279,
+      "learning_rate": 0.00019980700079598996,
+      "loss": 2.0394,
+      "step": 62
+    },
+    {
+      "epoch": 0.0034404139418679263,
+      "grad_norm": 9.552716255187988,
+      "learning_rate": 0.00019979950887949758,
+      "loss": 1.6357,
+      "step": 63
+    },
+    {
+      "epoch": 0.003495023686976941,
+      "grad_norm": 7.435667991638184,
+      "learning_rate": 0.00019979187445248498,
+      "loss": 2.2283,
+      "step": 64
+    },
+    {
+      "epoch": 0.003549633432085956,
+      "grad_norm": 6.557869911193848,
+      "learning_rate": 0.00019978409752585376,
+      "loss": 1.673,
+      "step": 65
+    },
+    {
+      "epoch": 0.0036042431771949705,
+      "grad_norm": 8.399250030517578,
+      "learning_rate": 0.00019977617811070924,
+      "loss": 1.8152,
+      "step": 66
+    },
+    {
+      "epoch": 0.003658852922303985,
+      "grad_norm": 6.870660305023193,
+      "learning_rate": 0.00019976811621836,
+      "loss": 1.8198,
+      "step": 67
+    },
+    {
+      "epoch": 0.0037134626674129997,
+      "grad_norm": 8.202585220336914,
+      "learning_rate": 0.00019975991186031825,
+      "loss": 1.9876,
+      "step": 68
+    },
+    {
+      "epoch": 0.0037680724125220147,
+      "grad_norm": 7.708743572235107,
+      "learning_rate": 0.00019975156504829946,
+      "loss": 2.0953,
+      "step": 69
+    },
+    {
+      "epoch": 0.0038226821576310293,
+      "grad_norm": 6.792536735534668,
+      "learning_rate": 0.00019974307579422268,
+      "loss": 2.0132,
+      "step": 70
+    },
+    {
+      "epoch": 0.003877291902740044,
+      "grad_norm": 6.443492412567139,
+      "learning_rate": 0.0001997344441102103,
+      "loss": 1.7874,
+      "step": 71
+    },
+    {
+      "epoch": 0.003931901647849059,
+      "grad_norm": 8.258990287780762,
+      "learning_rate": 0.00019972567000858807,
+      "loss": 1.85,
+      "step": 72
+    },
+    {
+      "epoch": 0.003986511392958073,
+      "grad_norm": 6.137606143951416,
+      "learning_rate": 0.00019971675350188508,
+      "loss": 1.8521,
+      "step": 73
+    },
+    {
+      "epoch": 0.004041121138067088,
+      "grad_norm": 6.298130989074707,
+      "learning_rate": 0.00019970769460283386,
+      "loss": 1.7568,
+      "step": 74
+    },
+    {
+      "epoch": 0.004095730883176103,
+      "grad_norm": 5.564772605895996,
+      "learning_rate": 0.00019969849332437028,
+      "loss": 1.5292,
+      "step": 75
+    },
+    {
+      "epoch": 0.004150340628285117,
+      "grad_norm": 7.886135578155518,
+      "learning_rate": 0.00019968914967963337,
+      "loss": 1.752,
+      "step": 76
+    },
+    {
+      "epoch": 0.004204950373394132,
+      "grad_norm": 7.4699506759643555,
+      "learning_rate": 0.00019967966368196564,
+      "loss": 1.897,
+      "step": 77
+    },
+    {
+      "epoch": 0.004259560118503147,
+      "grad_norm": 7.655289173126221,
+      "learning_rate": 0.00019967003534491275,
+      "loss": 1.9571,
+      "step": 78
+    },
+    {
+      "epoch": 0.0043141698636121615,
+      "grad_norm": 7.319615364074707,
+      "learning_rate": 0.00019966026468222364,
+      "loss": 1.9617,
+      "step": 79
+    },
+    {
+      "epoch": 0.0043687796087211765,
+      "grad_norm": 6.048542022705078,
+      "learning_rate": 0.00019965035170785062,
+      "loss": 1.553,
+      "step": 80
+    },
+    {
+      "epoch": 0.004423389353830191,
+      "grad_norm": 7.039748191833496,
+      "learning_rate": 0.00019964029643594895,
+      "loss": 1.7788,
+      "step": 81
+    },
+    {
+      "epoch": 0.004477999098939206,
+      "grad_norm": 8.082524299621582,
+      "learning_rate": 0.00019963009888087732,
+      "loss": 1.6795,
+      "step": 82
+    },
+    {
+      "epoch": 0.004532608844048221,
+      "grad_norm": 6.804952144622803,
+      "learning_rate": 0.0001996197590571975,
+      "loss": 1.6676,
+      "step": 83
+    },
+    {
+      "epoch": 0.004587218589157235,
+      "grad_norm": 118.06906127929688,
+      "learning_rate": 0.00019960927697967442,
+      "loss": 2.0885,
+      "step": 84
+    },
+    {
+      "epoch": 0.00464182833426625,
+      "grad_norm": 6.40289306640625,
+      "learning_rate": 0.00019959865266327618,
+      "loss": 1.5842,
+      "step": 85
+    },
+    {
+      "epoch": 0.004696438079375265,
+      "grad_norm": 7.656285762786865,
+      "learning_rate": 0.00019958788612317393,
+      "loss": 2.0125,
+      "step": 86
+    },
+    {
+      "epoch": 0.004751047824484279,
+      "grad_norm": 7.363048553466797,
+      "learning_rate": 0.00019957697737474196,
+      "loss": 1.955,
+      "step": 87
+    },
+    {
+      "epoch": 0.004805657569593294,
+      "grad_norm": 6.324362754821777,
+      "learning_rate": 0.00019956592643355763,
+      "loss": 1.6217,
+      "step": 88
+    },
+    {
+      "epoch": 0.004860267314702308,
+      "grad_norm": 5.8138203620910645,
+      "learning_rate": 0.0001995547333154013,
+      "loss": 1.8268,
+      "step": 89
+    },
+    {
+      "epoch": 0.004914877059811323,
+      "grad_norm": 4.656023979187012,
+      "learning_rate": 0.00019954339803625643,
+      "loss": 1.4257,
+      "step": 90
+    },
+    {
+      "epoch": 0.004969486804920338,
+      "grad_norm": 7.066349983215332,
+      "learning_rate": 0.00019953192061230942,
+      "loss": 1.7502,
+      "step": 91
+    },
+    {
+      "epoch": 0.005024096550029352,
+      "grad_norm": 6.3639092445373535,
+      "learning_rate": 0.00019952030105994964,
+      "loss": 1.8118,
+      "step": 92
+    },
+    {
+      "epoch": 0.005078706295138367,
+      "grad_norm": 5.50360107421875,
+      "learning_rate": 0.00019950853939576947,
+      "loss": 1.3473,
+      "step": 93
+    },
+    {
+      "epoch": 0.005133316040247382,
+      "grad_norm": 6.363309860229492,
+      "learning_rate": 0.00019949663563656418,
+      "loss": 1.7115,
+      "step": 94
+    },
+    {
+      "epoch": 0.005187925785356397,
+      "grad_norm": 8.41908073425293,
+      "learning_rate": 0.00019948458979933195,
+      "loss": 1.9494,
+      "step": 95
+    },
+    {
+      "epoch": 0.005242535530465412,
+      "grad_norm": 6.335050106048584,
+      "learning_rate": 0.0001994724019012739,
+      "loss": 1.6338,
+      "step": 96
+    },
+    {
+      "epoch": 0.005297145275574427,
+      "grad_norm": 5.774214267730713,
+      "learning_rate": 0.00019946007195979388,
+      "loss": 1.5344,
+      "step": 97
+    },
+    {
+      "epoch": 0.005351755020683441,
+      "grad_norm": 5.569876670837402,
+      "learning_rate": 0.00019944759999249872,
+      "loss": 1.4151,
+      "step": 98
+    },
+    {
+      "epoch": 0.005406364765792456,
+      "grad_norm": 5.1033220291137695,
+      "learning_rate": 0.00019943498601719797,
+      "loss": 1.3845,
+      "step": 99
+    },
+    {
+      "epoch": 0.00546097451090147,
+      "grad_norm": 6.493510723114014,
+      "learning_rate": 0.00019942223005190398,
+      "loss": 1.6274,
+      "step": 100
+    },
+    {
+      "epoch": 0.005515584256010485,
+      "grad_norm": 9.078402519226074,
+      "learning_rate": 0.00019940933211483187,
+      "loss": 2.0351,
+      "step": 101
+    },
+    {
+      "epoch": 0.0055701940011195,
+      "grad_norm": 9.098546981811523,
+      "learning_rate": 0.00019939629222439945,
+      "loss": 1.6705,
+      "step": 102
+    },
+    {
+      "epoch": 0.005624803746228514,
+      "grad_norm": 5.919679164886475,
+      "learning_rate": 0.00019938311039922736,
+      "loss": 1.3473,
+      "step": 103
+    },
+    {
+      "epoch": 0.005679413491337529,
+      "grad_norm": 8.595351219177246,
+      "learning_rate": 0.00019936978665813872,
+      "loss": 1.7132,
+      "step": 104
+    },
+    {
+      "epoch": 0.005734023236446544,
+      "grad_norm": 8.662528991699219,
+      "learning_rate": 0.00019935632102015944,
+      "loss": 2.0065,
+      "step": 105
+    },
+    {
+      "epoch": 0.005788632981555558,
+      "grad_norm": 6.772997856140137,
+      "learning_rate": 0.00019934271350451809,
+      "loss": 1.7575,
+      "step": 106
+    },
+    {
+      "epoch": 0.005843242726664573,
+      "grad_norm": 8.997614860534668,
+      "learning_rate": 0.00019932896413064568,
+      "loss": 1.602,
+      "step": 107
+    },
+    {
+      "epoch": 0.005897852471773588,
+      "grad_norm": 9.720958709716797,
+      "learning_rate": 0.0001993150729181759,
+      "loss": 1.919,
+      "step": 108
+    },
+    {
+      "epoch": 0.0059524622168826025,
+      "grad_norm": 6.682377338409424,
+      "learning_rate": 0.000199301039886945,
+      "loss": 1.5043,
+      "step": 109
+    },
+    {
+      "epoch": 0.0060070719619916176,
+      "grad_norm": 6.788079261779785,
+      "learning_rate": 0.00019928686505699163,
+      "loss": 1.4602,
+      "step": 110
+    },
+    {
+      "epoch": 0.006061681707100632,
+      "grad_norm": 8.357623100280762,
+      "learning_rate": 0.0001992725484485571,
+      "loss": 1.8856,
+      "step": 111
+    },
+    {
+      "epoch": 0.006116291452209647,
+      "grad_norm": 6.496567726135254,
+      "learning_rate": 0.00019925809008208495,
+      "loss": 1.8505,
+      "step": 112
+    },
+    {
+      "epoch": 0.006170901197318662,
+      "grad_norm": 6.439908504486084,
+      "learning_rate": 0.00019924348997822134,
+      "loss": 1.5902,
+      "step": 113
+    },
+    {
+      "epoch": 0.006225510942427676,
+      "grad_norm": 6.017791271209717,
+      "learning_rate": 0.00019922874815781478,
+      "loss": 1.309,
+      "step": 114
+    },
+    {
+      "epoch": 0.006280120687536691,
+      "grad_norm": 6.003462314605713,
+      "learning_rate": 0.00019921386464191604,
+      "loss": 1.8161,
+      "step": 115
+    },
+    {
+      "epoch": 0.006334730432645706,
+      "grad_norm": 6.161805629730225,
+      "learning_rate": 0.00019919883945177834,
+      "loss": 1.3689,
+      "step": 116
+    },
+    {
+      "epoch": 0.00638934017775472,
+      "grad_norm": 7.63073205947876,
+      "learning_rate": 0.00019918367260885722,
+      "loss": 1.8837,
+      "step": 117
+    },
+    {
+      "epoch": 0.006443949922863735,
+      "grad_norm": 7.145249843597412,
+      "learning_rate": 0.00019916836413481036,
+      "loss": 2.0012,
+      "step": 118
+    },
+    {
+      "epoch": 0.00649855966797275,
+      "grad_norm": 8.52342700958252,
+      "learning_rate": 0.00019915291405149787,
+      "loss": 1.7905,
+      "step": 119
+    },
+    {
+      "epoch": 0.006553169413081764,
+      "grad_norm": 7.2071757316589355,
+      "learning_rate": 0.0001991373223809819,
+      "loss": 1.9477,
+      "step": 120
+    },
+    {
+      "epoch": 0.006607779158190779,
+      "grad_norm": 6.978395938873291,
+      "learning_rate": 0.0001991215891455269,
+      "loss": 1.4861,
+      "step": 121
+    },
+    {
+      "epoch": 0.0066623889032997935,
+      "grad_norm": 67.82074737548828,
+      "learning_rate": 0.0001991057143675994,
+      "loss": 1.681,
+      "step": 122
+    },
+    {
+      "epoch": 0.0067169986484088085,
+      "grad_norm": 5.937620639801025,
+      "learning_rate": 0.0001990896980698681,
+      "loss": 1.953,
+      "step": 123
+    },
+    {
+      "epoch": 0.0067716083935178235,
+      "grad_norm": 6.759126663208008,
+      "learning_rate": 0.00019907354027520375,
+      "loss": 1.7718,
+      "step": 124
+    },
+    {
+      "epoch": 0.006826218138626838,
+      "grad_norm": 7.827403545379639,
+      "learning_rate": 0.0001990572410066792,
+      "loss": 2.0704,
+      "step": 125
+    },
+    {
+      "epoch": 0.006880827883735853,
+      "grad_norm": 6.467216968536377,
+      "learning_rate": 0.00019904080028756925,
+      "loss": 1.6904,
+      "step": 126
+    },
+    {
+      "epoch": 0.006935437628844868,
+      "grad_norm": 5.911613941192627,
+      "learning_rate": 0.0001990242181413507,
+      "loss": 1.128,
+      "step": 127
+    },
+    {
+      "epoch": 0.006990047373953882,
+      "grad_norm": 6.809331893920898,
+      "learning_rate": 0.00019900749459170238,
+      "loss": 1.9146,
+      "step": 128
+    },
+    {
+      "epoch": 0.007044657119062897,
+      "grad_norm": 7.211541652679443,
+      "learning_rate": 0.000198990629662505,
+      "loss": 2.0752,
+      "step": 129
+    },
+    {
+      "epoch": 0.007099266864171912,
+      "grad_norm": 6.313450336456299,
+      "learning_rate": 0.00019897362337784107,
+      "loss": 1.6653,
+      "step": 130
+    },
+    {
+      "epoch": 0.007153876609280926,
+      "grad_norm": 5.75939416885376,
+      "learning_rate": 0.00019895647576199506,
+      "loss": 1.6187,
+      "step": 131
+    },
+    {
+      "epoch": 0.007208486354389941,
+      "grad_norm": 7.255271911621094,
+      "learning_rate": 0.0001989391868394532,
+      "loss": 1.8374,
+      "step": 132
+    },
+    {
+      "epoch": 0.007263096099498955,
+      "grad_norm": 7.261573791503906,
+      "learning_rate": 0.0001989217566349036,
+      "loss": 1.6846,
+      "step": 133
+    },
+    {
+      "epoch": 0.00731770584460797,
+      "grad_norm": 7.004489898681641,
+      "learning_rate": 0.00019890418517323592,
+      "loss": 1.6945,
+      "step": 134
+    },
+    {
+      "epoch": 0.007372315589716985,
+      "grad_norm": 6.714308738708496,
+      "learning_rate": 0.00019888647247954172,
+      "loss": 1.3957,
+      "step": 135
+    },
+    {
+      "epoch": 0.007426925334825999,
+      "grad_norm": 7.432577133178711,
+      "learning_rate": 0.00019886861857911417,
+      "loss": 1.5253,
+      "step": 136
+    },
+    {
+      "epoch": 0.007481535079935014,
+      "grad_norm": 5.932303428649902,
+      "learning_rate": 0.00019885062349744802,
+      "loss": 1.3154,
+      "step": 137
+    },
+    {
+      "epoch": 0.0075361448250440295,
+      "grad_norm": 8.55103874206543,
+      "learning_rate": 0.00019883248726023972,
+      "loss": 1.7869,
+      "step": 138
+    },
+    {
+      "epoch": 0.007590754570153044,
+      "grad_norm": 4.898272514343262,
+      "learning_rate": 0.00019881420989338721,
+      "loss": 1.1577,
+      "step": 139
+    },
+    {
+      "epoch": 0.007645364315262059,
+      "grad_norm": 7.714294910430908,
+      "learning_rate": 0.00019879579142299,
+      "loss": 1.9076,
+      "step": 140
+    },
+    {
+      "epoch": 0.007699974060371074,
+      "grad_norm": 7.234163761138916,
+      "learning_rate": 0.00019877723187534908,
+      "loss": 2.0401,
+      "step": 141
+    },
+    {
+      "epoch": 0.007754583805480088,
+      "grad_norm": 5.768650531768799,
+      "learning_rate": 0.00019875853127696692,
+      "loss": 1.6223,
+      "step": 142
+    },
+    {
+      "epoch": 0.007809193550589103,
+      "grad_norm": 6.646040439605713,
+      "learning_rate": 0.0001987396896545473,
+      "loss": 1.3798,
+      "step": 143
+    },
+    {
+      "epoch": 0.007863803295698118,
+      "grad_norm": 7.1445465087890625,
+      "learning_rate": 0.00019872070703499554,
+      "loss": 1.9528,
+      "step": 144
+    },
+    {
+      "epoch": 0.007918413040807133,
+      "grad_norm": 7.658993721008301,
+      "learning_rate": 0.0001987015834454182,
+      "loss": 1.9569,
+      "step": 145
+    },
+    {
+      "epoch": 0.007973022785916146,
+      "grad_norm": 6.749288558959961,
+      "learning_rate": 0.0001986823189131231,
+      "loss": 1.5658,
+      "step": 146
+    },
+    {
+      "epoch": 0.008027632531025161,
+      "grad_norm": 8.621613502502441,
+      "learning_rate": 0.0001986629134656194,
+      "loss": 1.6405,
+      "step": 147
+    },
+    {
+      "epoch": 0.008082242276134176,
+      "grad_norm": 5.518367290496826,
+      "learning_rate": 0.00019864336713061752,
+      "loss": 1.4671,
+      "step": 148
+    },
+    {
+      "epoch": 0.008136852021243191,
+      "grad_norm": 7.534793376922607,
+      "learning_rate": 0.00019862367993602896,
+      "loss": 1.6298,
+      "step": 149
+    },
+    {
+      "epoch": 0.008191461766352206,
+      "grad_norm": 6.737891674041748,
+      "learning_rate": 0.0001986038519099664,
+      "loss": 1.8377,
+      "step": 150
+    },
+    {
+      "epoch": 0.008191461766352206,
+      "eval_loss": 0.39922699332237244,
+      "eval_runtime": 1197.5618,
+      "eval_samples_per_second": 4.175,
+      "eval_steps_per_second": 2.088,
+      "step": 150
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 2639,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 150,
+  "stateful_callbacks": {
+    "EarlyStoppingCallback": {
+      "args": {
+        "early_stopping_patience": 4,
+        "early_stopping_threshold": 0.0
+      },
+      "attributes": {
+        "early_stopping_patience_counter": 0
+      }
+    },
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.0750372542519706e+17,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:242dd2672a9181078dc53e7c149247ebb4c4f544a6983f9282c9bd69264de3d4
+size 6776