Upload folder using huggingface_hub

README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2

adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "o_proj",
+    "gate_proj",
+    "k_proj",
+    "v_proj",
+    "up_proj",
+    "down_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9ab4de5ee880067b2646f846b6a405e3bcecc1a89061d31654769b653a8f7bfe
+size 335604696

chat_template.jinja

@@ -0,0 +1,87 @@
+{%- if messages[0]["role"] == "system" %}
+    {%- set system_message = messages[0]["content"] %}
+    {%- set loop_messages = messages[1:] %}
+{%- else %}
+    {%- set loop_messages = messages %}
+{%- endif %}
+{%- if not tools is defined %}
+    {%- set tools = none %}
+{%- endif %}
+{%- set user_messages = loop_messages | selectattr("role", "equalto", "user") | list %}
+
+{#- This block checks for alternating user/assistant messages, skipping tool calling messages #}
+{%- set ns = namespace() %}
+{%- set ns.index = 0 %}
+{%- for message in loop_messages %}
+    {%- if not (message.role == "tool" or message.role == "tool_results" or (message.tool_calls is defined and message.tool_calls is not none)) %}
+        {%- if (message["role"] == "user") != (ns.index % 2 == 0) %}
+            {{- raise_exception("After the optional system message, conversation roles must alternate user/assistant/user/assistant/...") }}
+        {%- endif %}
+        {%- set ns.index = ns.index + 1 %}
+    {%- endif %}
+{%- endfor %}
+
+{{- bos_token }}
+{%- for message in loop_messages %}
+    {%- if message["role"] == "user" %}
+        {%- if tools is not none and (message == user_messages[-1]) %}
+            {{- "[AVAILABLE_TOOLS] [" }}
+            {%- for tool in tools %}
+                {%- set tool = tool.function %}
+                {{- '{"type": "function", "function": {' }}
+                {%- for key, val in tool.items() if key != "return" %}
+                    {%- if val is string %}
+                        {{- '"' + key + '": "' + val + '"' }}
+                    {%- else %}
+                        {{- '"' + key + '": ' + val|tojson }}
+                    {%- endif %}
+                    {%- if not loop.last %}
+                        {{- ", " }}
+                    {%- endif %}
+                {%- endfor %}
+                {{- "}}" }}
+                {%- if not loop.last %}
+                    {{- ", " }}
+                {%- else %}
+                    {{- "]" }}
+                {%- endif %}
+            {%- endfor %}
+            {{- "[/AVAILABLE_TOOLS]" }}
+            {%- endif %}
+        {%- if loop.last and system_message is defined %}
+            {{- "[INST] " + system_message + "\n\n" + message["content"] + "[/INST]" }}
+        {%- else %}
+            {{- "[INST] " + message["content"] + "[/INST]" }}
+        {%- endif %}
+    {%- elif message.tool_calls is defined and message.tool_calls is not none %}
+        {{- "[TOOL_CALLS] [" }}
+        {%- for tool_call in message.tool_calls %}
+            {%- set out = tool_call.function|tojson %}
+            {{- out[:-1] }}
+            {%- if not tool_call.id is defined or tool_call.id|length != 9 %}
+                {{- raise_exception("Tool call IDs should be alphanumeric strings with length 9!") }}
+            {%- endif %}
+            {{- ', "id": "' + tool_call.id + '"}' }}
+            {%- if not loop.last %}
+                {{- ", " }}
+            {%- else %}
+                {{- "]" + eos_token }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif message["role"] == "assistant" %}
+        {{- " " + message["content"]|trim + eos_token}}
+    {%- elif message["role"] == "tool_results" or message["role"] == "tool" %}
+        {%- if message.content is defined and message.content.content is defined %}
+            {%- set content = message.content.content %}
+        {%- else %}
+            {%- set content = message.content %}
+        {%- endif %}
+        {{- '[TOOL_RESULTS] {"content": ' + content|string + ", " }}
+        {%- if not message.tool_call_id is defined or message.tool_call_id|length != 9 %}
+            {{- raise_exception("Tool call IDs should be alphanumeric strings with length 9!") }}
+        {%- endif %}
+        {{- '"call_id": "' + message.tool_call_id + '"}[/TOOL_RESULTS]' }}
+    {%- else %}
+        {{- raise_exception("Only user and assistant roles are supported, with the exception of an initial optional system message!") }}
+    {%- endif %}
+{%- endfor %}

checkpoint-1245/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2

checkpoint-1245/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "o_proj",
+    "gate_proj",
+    "k_proj",
+    "v_proj",
+    "up_proj",
+    "down_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-1245/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9ab4de5ee880067b2646f846b6a405e3bcecc1a89061d31654769b653a8f7bfe
+size 335604696

checkpoint-1245/chat_template.jinja

@@ -0,0 +1,87 @@
+{%- if messages[0]["role"] == "system" %}
+    {%- set system_message = messages[0]["content"] %}
+    {%- set loop_messages = messages[1:] %}
+{%- else %}
+    {%- set loop_messages = messages %}
+{%- endif %}
+{%- if not tools is defined %}
+    {%- set tools = none %}
+{%- endif %}
+{%- set user_messages = loop_messages | selectattr("role", "equalto", "user") | list %}
+
+{#- This block checks for alternating user/assistant messages, skipping tool calling messages #}
+{%- set ns = namespace() %}
+{%- set ns.index = 0 %}
+{%- for message in loop_messages %}
+    {%- if not (message.role == "tool" or message.role == "tool_results" or (message.tool_calls is defined and message.tool_calls is not none)) %}
+        {%- if (message["role"] == "user") != (ns.index % 2 == 0) %}
+            {{- raise_exception("After the optional system message, conversation roles must alternate user/assistant/user/assistant/...") }}
+        {%- endif %}
+        {%- set ns.index = ns.index + 1 %}
+    {%- endif %}
+{%- endfor %}
+
+{{- bos_token }}
+{%- for message in loop_messages %}
+    {%- if message["role"] == "user" %}
+        {%- if tools is not none and (message == user_messages[-1]) %}
+            {{- "[AVAILABLE_TOOLS] [" }}
+            {%- for tool in tools %}
+                {%- set tool = tool.function %}
+                {{- '{"type": "function", "function": {' }}
+                {%- for key, val in tool.items() if key != "return" %}
+                    {%- if val is string %}
+                        {{- '"' + key + '": "' + val + '"' }}
+                    {%- else %}
+                        {{- '"' + key + '": ' + val|tojson }}
+                    {%- endif %}
+                    {%- if not loop.last %}
+                        {{- ", " }}
+                    {%- endif %}
+                {%- endfor %}
+                {{- "}}" }}
+                {%- if not loop.last %}
+                    {{- ", " }}
+                {%- else %}
+                    {{- "]" }}
+                {%- endif %}
+            {%- endfor %}
+            {{- "[/AVAILABLE_TOOLS]" }}
+            {%- endif %}
+        {%- if loop.last and system_message is defined %}
+            {{- "[INST] " + system_message + "\n\n" + message["content"] + "[/INST]" }}
+        {%- else %}
+            {{- "[INST] " + message["content"] + "[/INST]" }}
+        {%- endif %}
+    {%- elif message.tool_calls is defined and message.tool_calls is not none %}
+        {{- "[TOOL_CALLS] [" }}
+        {%- for tool_call in message.tool_calls %}
+            {%- set out = tool_call.function|tojson %}
+            {{- out[:-1] }}
+            {%- if not tool_call.id is defined or tool_call.id|length != 9 %}
+                {{- raise_exception("Tool call IDs should be alphanumeric strings with length 9!") }}
+            {%- endif %}
+            {{- ', "id": "' + tool_call.id + '"}' }}
+            {%- if not loop.last %}
+                {{- ", " }}
+            {%- else %}
+                {{- "]" + eos_token }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif message["role"] == "assistant" %}
+        {{- " " + message["content"]|trim + eos_token}}
+    {%- elif message["role"] == "tool_results" or message["role"] == "tool" %}
+        {%- if message.content is defined and message.content.content is defined %}
+            {%- set content = message.content.content %}
+        {%- else %}
+            {%- set content = message.content %}
+        {%- endif %}
+        {{- '[TOOL_RESULTS] {"content": ' + content|string + ", " }}
+        {%- if not message.tool_call_id is defined or message.tool_call_id|length != 9 %}
+            {{- raise_exception("Tool call IDs should be alphanumeric strings with length 9!") }}
+        {%- endif %}
+        {{- '"call_id": "' + message.tool_call_id + '"}[/TOOL_RESULTS]' }}
+    {%- else %}
+        {{- raise_exception("Only user and assistant roles are supported, with the exception of an initial optional system message!") }}
+    {%- endif %}
+{%- endfor %}

checkpoint-1245/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ddb8b96ff4b013d34824312098c35c25ae487f82ca882df6e967a86e74710c41
+size 671365003

checkpoint-1245/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3facc22bab6b04e596ed724b1b9b557333baba1218d3c203ddc14b61d0749f5f
+size 14645

checkpoint-1245/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:82fcd00123c3c69d4e7b09d8e96247aa0926ecb0862a7624726a4095234b5d76
+size 1465

checkpoint-1245/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-1245/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

checkpoint-1245/trainer_state.json

@@ -0,0 +1,475 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 3.0,
+  "eval_steps": 500,
+  "global_step": 1245,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.060350030175015085,
+      "grad_norm": 0.6138771176338196,
+      "learning_rate": 0.0001894736842105263,
+      "loss": 1.3796,
+      "mean_token_accuracy": 0.7022829699516296,
+      "num_tokens": 154999.0,
+      "step": 25
+    },
+    {
+      "epoch": 0.12070006035003017,
+      "grad_norm": 0.6403661370277405,
+      "learning_rate": 0.00029993852448555923,
+      "loss": 0.7408,
+      "mean_token_accuracy": 0.8062405365705491,
+      "num_tokens": 274882.0,
+      "step": 50
+    },
+    {
+      "epoch": 0.18105009052504525,
+      "grad_norm": 0.5453419089317322,
+      "learning_rate": 0.00029934198818572623,
+      "loss": 0.4995,
+      "mean_token_accuracy": 0.8602466344833374,
+      "num_tokens": 429270.0,
+      "step": 75
+    },
+    {
+      "epoch": 0.24140012070006034,
+      "grad_norm": 0.9569323062896729,
+      "learning_rate": 0.0002981133400718627,
+      "loss": 0.3791,
+      "mean_token_accuracy": 0.8920469450950622,
+      "num_tokens": 547533.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.30175015087507545,
+      "grad_norm": 0.4823763966560364,
+      "learning_rate": 0.0002962577805768642,
+      "loss": 0.2865,
+      "mean_token_accuracy": 0.919411507844925,
+      "num_tokens": 700414.0,
+      "step": 125
+    },
+    {
+      "epoch": 0.3621001810500905,
+      "grad_norm": 0.5946052670478821,
+      "learning_rate": 0.00029378316362776546,
+      "loss": 0.3033,
+      "mean_token_accuracy": 0.9159089189767837,
+      "num_tokens": 818685.0,
+      "step": 150
+    },
+    {
+      "epoch": 0.4224502112251056,
+      "grad_norm": 0.513756513595581,
+      "learning_rate": 0.0002906999634028451,
+      "loss": 0.2095,
+      "mean_token_accuracy": 0.9410657143592834,
+      "num_tokens": 973411.0,
+      "step": 175
+    },
+    {
+      "epoch": 0.4828002414001207,
+      "grad_norm": 0.5572422742843628,
+      "learning_rate": 0.0002870212299981334,
+      "loss": 0.1986,
+      "mean_token_accuracy": 0.9448752331733704,
+      "num_tokens": 1091985.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.5431502715751357,
+      "grad_norm": 0.3195100724697113,
+      "learning_rate": 0.00028276253419097193,
+      "loss": 0.1613,
+      "mean_token_accuracy": 0.9562808114290238,
+      "num_tokens": 1249334.0,
+      "step": 225
+    },
+    {
+      "epoch": 0.6035003017501509,
+      "grad_norm": 0.8021010756492615,
+      "learning_rate": 0.00027794190153442033,
+      "loss": 0.161,
+      "mean_token_accuracy": 0.9557685399055481,
+      "num_tokens": 1367322.0,
+      "step": 250
+    },
+    {
+      "epoch": 0.663850331925166,
+      "grad_norm": 0.4245581328868866,
+      "learning_rate": 0.00027257973606146575,
+      "loss": 0.1414,
+      "mean_token_accuracy": 0.9619652438163757,
+      "num_tokens": 1523489.0,
+      "step": 275
+    },
+    {
+      "epoch": 0.724200362100181,
+      "grad_norm": 0.7518550157546997,
+      "learning_rate": 0.0002666987339219681,
+      "loss": 0.1313,
+      "mean_token_accuracy": 0.9653391098976135,
+      "num_tokens": 1644051.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.7845503922751962,
+      "grad_norm": 0.5166082978248596,
+      "learning_rate": 0.0002603237873178853,
+      "loss": 0.1179,
+      "mean_token_accuracy": 0.9685099506378174,
+      "num_tokens": 1799510.0,
+      "step": 325
+    },
+    {
+      "epoch": 0.8449004224502112,
+      "grad_norm": 0.8754676580429077,
+      "learning_rate": 0.0002534818791433866,
+      "loss": 0.1172,
+      "mean_token_accuracy": 0.9700138258934021,
+      "num_tokens": 1918554.0,
+      "step": 350
+    },
+    {
+      "epoch": 0.9052504526252263,
+      "grad_norm": 0.5072232484817505,
+      "learning_rate": 0.00024620196877580576,
+      "loss": 0.1087,
+      "mean_token_accuracy": 0.9706882643699646,
+      "num_tokens": 2071557.0,
+      "step": 375
+    },
+    {
+      "epoch": 0.9656004828002414,
+      "grad_norm": 0.44057291746139526,
+      "learning_rate": 0.00023851486950083892,
+      "loss": 0.1102,
+      "mean_token_accuracy": 0.9717801564931869,
+      "num_tokens": 2188325.0,
+      "step": 400
+    },
+    {
+      "epoch": 1.024140012070006,
+      "grad_norm": 0.28425154089927673,
+      "learning_rate": 0.00023045311809080567,
+      "loss": 0.0969,
+      "mean_token_accuracy": 0.9745202402478641,
+      "num_tokens": 2335834.0,
+      "step": 425
+    },
+    {
+      "epoch": 1.0844900422450212,
+      "grad_norm": 0.27836862206459045,
+      "learning_rate": 0.00022205083708799942,
+      "loss": 0.0775,
+      "mean_token_accuracy": 0.9795319002866745,
+      "num_tokens": 2475640.0,
+      "step": 450
+    },
+    {
+      "epoch": 1.1448400724200363,
+      "grad_norm": 0.25006264448165894,
+      "learning_rate": 0.0002133435903760353,
+      "loss": 0.0926,
+      "mean_token_accuracy": 0.9758930206298828,
+      "num_tokens": 2613115.0,
+      "step": 475
+    },
+    {
+      "epoch": 1.2051901025950513,
+      "grad_norm": 0.3176310658454895,
+      "learning_rate": 0.0002043682326505094,
+      "loss": 0.0687,
+      "mean_token_accuracy": 0.9820343238115311,
+      "num_tokens": 2749269.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.2655401327700664,
+      "grad_norm": 0.20867092907428741,
+      "learning_rate": 0.000195162753426108,
+      "loss": 0.0877,
+      "mean_token_accuracy": 0.9773262399435043,
+      "num_tokens": 2883349.0,
+      "step": 525
+    },
+    {
+      "epoch": 1.3258901629450814,
+      "grad_norm": 0.25837627053260803,
+      "learning_rate": 0.00018576611624042852,
+      "loss": 0.0674,
+      "mean_token_accuracy": 0.9824212837219238,
+      "num_tokens": 3020654.0,
+      "step": 550
+    },
+    {
+      "epoch": 1.3862401931200965,
+      "grad_norm": 0.1842041015625,
+      "learning_rate": 0.00017621809373510641,
+      "loss": 0.0819,
+      "mean_token_accuracy": 0.979530062675476,
+      "num_tokens": 3157335.0,
+      "step": 575
+    },
+    {
+      "epoch": 1.4465902232951118,
+      "grad_norm": 0.2597507834434509,
+      "learning_rate": 0.00016655909931229048,
+      "loss": 0.0659,
+      "mean_token_accuracy": 0.9828319466114044,
+      "num_tokens": 3294681.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.5069402534701268,
+      "grad_norm": 0.17711229622364044,
+      "learning_rate": 0.00015683001607900553,
+      "loss": 0.0842,
+      "mean_token_accuracy": 0.978652862906456,
+      "num_tokens": 3430142.0,
+      "step": 625
+    },
+    {
+      "epoch": 1.567290283645142,
+      "grad_norm": 0.13509850203990936,
+      "learning_rate": 0.00014707202380342108,
+      "loss": 0.0573,
+      "mean_token_accuracy": 0.9850119787454605,
+      "num_tokens": 3567792.0,
+      "step": 650
+    },
+    {
+      "epoch": 1.627640313820157,
+      "grad_norm": 0.14440615475177765,
+      "learning_rate": 0.00013732642461545747,
+      "loss": 0.0758,
+      "mean_token_accuracy": 0.9804488325119018,
+      "num_tokens": 3703772.0,
+      "step": 675
+    },
+    {
+      "epoch": 1.687990343995172,
+      "grad_norm": 0.12784038484096527,
+      "learning_rate": 0.00012763446818947865,
+      "loss": 0.0557,
+      "mean_token_accuracy": 0.9857299762964249,
+      "num_tokens": 3841338.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.748340374170187,
+      "grad_norm": 0.1049482449889183,
+      "learning_rate": 0.00011803717714901029,
+      "loss": 0.0692,
+      "mean_token_accuracy": 0.9823172944784164,
+      "num_tokens": 3976256.0,
+      "step": 725
+    },
+    {
+      "epoch": 1.8086904043452021,
+      "grad_norm": 0.1479799747467041,
+      "learning_rate": 0.00010857517343248423,
+      "loss": 0.0544,
+      "mean_token_accuracy": 0.9862035536766052,
+      "num_tokens": 4113650.0,
+      "step": 750
+    },
+    {
+      "epoch": 1.8690404345202172,
+      "grad_norm": 0.10535310953855515,
+      "learning_rate": 9.9288506354941e-05,
+      "loss": 0.0688,
+      "mean_token_accuracy": 0.9820931887626648,
+      "num_tokens": 4250788.0,
+      "step": 775
+    },
+    {
+      "epoch": 1.9293904646952322,
+      "grad_norm": 0.1969379335641861,
+      "learning_rate": 9.021648309344443e-05,
+      "loss": 0.0523,
+      "mean_token_accuracy": 0.9861829555034638,
+      "num_tokens": 4388884.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.9897404948702473,
+      "grad_norm": 0.19566108286380768,
+      "learning_rate": 8.139750231370407e-05,
+      "loss": 0.0589,
+      "mean_token_accuracy": 0.9854754799604416,
+      "num_tokens": 4513974.0,
+      "step": 825
+    },
+    {
+      "epoch": 2.048280024140012,
+      "grad_norm": 0.08709706366062164,
+      "learning_rate": 7.28688916421049e-05,
+      "loss": 0.0557,
+      "mean_token_accuracy": 0.9854855641876299,
+      "num_tokens": 4657894.0,
+      "step": 850
+    },
+    {
+      "epoch": 2.1086300543150274,
+      "grad_norm": 0.32135921716690063,
+      "learning_rate": 6.466674967106751e-05,
+      "loss": 0.046,
+      "mean_token_accuracy": 0.9884958410263062,
+      "num_tokens": 4783309.0,
+      "step": 875
+    },
+    {
+      "epoch": 2.1689800844900424,
+      "grad_norm": 0.10706797987222672,
+      "learning_rate": 5.682579316647408e-05,
+      "loss": 0.0512,
+      "mean_token_accuracy": 0.9862526476383209,
+      "num_tokens": 4931681.0,
+      "step": 900
+    },
+    {
+      "epoch": 2.2293301146650575,
+      "grad_norm": 0.06649629771709442,
+      "learning_rate": 4.937921012387816e-05,
+      "loss": 0.0452,
+      "mean_token_accuracy": 0.9884600782394409,
+      "num_tokens": 5059261.0,
+      "step": 925
+    },
+    {
+      "epoch": 2.2896801448400725,
+      "grad_norm": 0.0824085995554924,
+      "learning_rate": 4.235851929545771e-05,
+      "loss": 0.0517,
+      "mean_token_accuracy": 0.9863192504644394,
+      "num_tokens": 5208555.0,
+      "step": 950
+    },
+    {
+      "epoch": 2.3500301750150876,
+      "grad_norm": 0.06256023794412613,
+      "learning_rate": 3.579343678228525e-05,
+      "loss": 0.0454,
+      "mean_token_accuracy": 0.9885574287176132,
+      "num_tokens": 5335514.0,
+      "step": 975
+    },
+    {
+      "epoch": 2.4103802051901027,
+      "grad_norm": 0.06850145757198334,
+      "learning_rate": 2.9711750256582538e-05,
+      "loss": 0.0525,
+      "mean_token_accuracy": 0.9856393277645111,
+      "num_tokens": 5486306.0,
+      "step": 1000
+    },
+    {
+      "epoch": 2.4707302353651177,
+      "grad_norm": 0.09066502004861832,
+      "learning_rate": 2.413920134633272e-05,
+      "loss": 0.045,
+      "mean_token_accuracy": 0.9883674955368043,
+      "num_tokens": 5612848.0,
+      "step": 1025
+    },
+    {
+      "epoch": 2.5310802655401328,
+      "grad_norm": 0.050995565950870514,
+      "learning_rate": 1.909937668007352e-05,
+      "loss": 0.0456,
+      "mean_token_accuracy": 0.9880671769380569,
+      "num_tokens": 5757364.0,
+      "step": 1050
+    },
+    {
+      "epoch": 2.591430295715148,
+      "grad_norm": 0.07163764536380768,
+      "learning_rate": 1.461360805304146e-05,
+      "loss": 0.0452,
+      "mean_token_accuracy": 0.9885901701450348,
+      "num_tokens": 5881265.0,
+      "step": 1075
+    },
+    {
+      "epoch": 2.651780325890163,
+      "grad_norm": 0.09139364957809448,
+      "learning_rate": 1.0700882137227434e-05,
+      "loss": 0.0479,
+      "mean_token_accuracy": 0.9873688906431198,
+      "num_tokens": 6030295.0,
+      "step": 1100
+    },
+    {
+      "epoch": 2.712130356065178,
+      "grad_norm": 0.10289441794157028,
+      "learning_rate": 7.377760117509834e-06,
+      "loss": 0.045,
+      "mean_token_accuracy": 0.9884589624404907,
+      "num_tokens": 6157143.0,
+      "step": 1125
+    },
+    {
+      "epoch": 2.772480386240193,
+      "grad_norm": 0.09236230701208115,
+      "learning_rate": 4.65830759401658e-06,
+      "loss": 0.046,
+      "mean_token_accuracy": 0.9879369300603866,
+      "num_tokens": 6303352.0,
+      "step": 1150
+    },
+    {
+      "epoch": 2.832830416415208,
+      "grad_norm": 0.04650961980223656,
+      "learning_rate": 2.554035047414732e-06,
+      "loss": 0.0444,
+      "mean_token_accuracy": 0.9887312567234039,
+      "num_tokens": 6428751.0,
+      "step": 1175
+    },
+    {
+      "epoch": 2.8931804465902236,
+      "grad_norm": 0.08433457463979721,
+      "learning_rate": 1.0738491191171372e-06,
+      "loss": 0.0458,
+      "mean_token_accuracy": 0.9878807801008225,
+      "num_tokens": 6576081.0,
+      "step": 1200
+    },
+    {
+      "epoch": 2.9535304767652386,
+      "grad_norm": 0.08487927168607712,
+      "learning_rate": 2.2401491261947456e-07,
+      "loss": 0.0428,
+      "mean_token_accuracy": 0.9895708286762237,
+      "num_tokens": 6701527.0,
+      "step": 1225
+    }
+  ],
+  "logging_steps": 25,
+  "max_steps": 1245,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 750,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.939824705454408e+17,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-1245/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1368312d6171cb3228a4505daeeebb2d50b85fd42516b8b22d1d350ef88234fe
+size 6033

checkpoint-750/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2

checkpoint-750/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "o_proj",
+    "gate_proj",
+    "k_proj",
+    "v_proj",
+    "up_proj",
+    "down_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-750/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d870137ccac6c8de0f64e8c5bc8962876a97b95e3dbed260d6598aad56474197
+size 335604696

checkpoint-750/chat_template.jinja

@@ -0,0 +1,87 @@
+{%- if messages[0]["role"] == "system" %}
+    {%- set system_message = messages[0]["content"] %}
+    {%- set loop_messages = messages[1:] %}
+{%- else %}
+    {%- set loop_messages = messages %}
+{%- endif %}
+{%- if not tools is defined %}
+    {%- set tools = none %}
+{%- endif %}
+{%- set user_messages = loop_messages | selectattr("role", "equalto", "user") | list %}
+
+{#- This block checks for alternating user/assistant messages, skipping tool calling messages #}
+{%- set ns = namespace() %}
+{%- set ns.index = 0 %}
+{%- for message in loop_messages %}
+    {%- if not (message.role == "tool" or message.role == "tool_results" or (message.tool_calls is defined and message.tool_calls is not none)) %}
+        {%- if (message["role"] == "user") != (ns.index % 2 == 0) %}
+            {{- raise_exception("After the optional system message, conversation roles must alternate user/assistant/user/assistant/...") }}
+        {%- endif %}
+        {%- set ns.index = ns.index + 1 %}
+    {%- endif %}
+{%- endfor %}
+
+{{- bos_token }}
+{%- for message in loop_messages %}
+    {%- if message["role"] == "user" %}
+        {%- if tools is not none and (message == user_messages[-1]) %}
+            {{- "[AVAILABLE_TOOLS] [" }}
+            {%- for tool in tools %}
+                {%- set tool = tool.function %}
+                {{- '{"type": "function", "function": {' }}
+                {%- for key, val in tool.items() if key != "return" %}
+                    {%- if val is string %}
+                        {{- '"' + key + '": "' + val + '"' }}
+                    {%- else %}
+                        {{- '"' + key + '": ' + val|tojson }}
+                    {%- endif %}
+                    {%- if not loop.last %}
+                        {{- ", " }}
+                    {%- endif %}
+                {%- endfor %}
+                {{- "}}" }}
+                {%- if not loop.last %}
+                    {{- ", " }}
+                {%- else %}
+                    {{- "]" }}
+                {%- endif %}
+            {%- endfor %}
+            {{- "[/AVAILABLE_TOOLS]" }}
+            {%- endif %}
+        {%- if loop.last and system_message is defined %}
+            {{- "[INST] " + system_message + "\n\n" + message["content"] + "[/INST]" }}
+        {%- else %}
+            {{- "[INST] " + message["content"] + "[/INST]" }}
+        {%- endif %}
+    {%- elif message.tool_calls is defined and message.tool_calls is not none %}
+        {{- "[TOOL_CALLS] [" }}
+        {%- for tool_call in message.tool_calls %}
+            {%- set out = tool_call.function|tojson %}
+            {{- out[:-1] }}
+            {%- if not tool_call.id is defined or tool_call.id|length != 9 %}
+                {{- raise_exception("Tool call IDs should be alphanumeric strings with length 9!") }}
+            {%- endif %}
+            {{- ', "id": "' + tool_call.id + '"}' }}
+            {%- if not loop.last %}
+                {{- ", " }}
+            {%- else %}
+                {{- "]" + eos_token }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif message["role"] == "assistant" %}
+        {{- " " + message["content"]|trim + eos_token}}
+    {%- elif message["role"] == "tool_results" or message["role"] == "tool" %}
+        {%- if message.content is defined and message.content.content is defined %}
+            {%- set content = message.content.content %}
+        {%- else %}
+            {%- set content = message.content %}
+        {%- endif %}
+        {{- '[TOOL_RESULTS] {"content": ' + content|string + ", " }}
+        {%- if not message.tool_call_id is defined or message.tool_call_id|length != 9 %}
+            {{- raise_exception("Tool call IDs should be alphanumeric strings with length 9!") }}
+        {%- endif %}
+        {{- '"call_id": "' + message.tool_call_id + '"}[/TOOL_RESULTS]' }}
+    {%- else %}
+        {{- raise_exception("Only user and assistant roles are supported, with the exception of an initial optional system message!") }}
+    {%- endif %}
+{%- endfor %}

checkpoint-750/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9711e9a9237a5ea3d281502d928c0c0e1a4eff82b30a36579b6054b27a09f663
+size 671365003

checkpoint-750/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d52543053744232d5775de24d3a7d074c700cbe44021221a4d6aa3d0848f1397
+size 14645

checkpoint-750/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f219c746dd03301eb7ed441bdaf27ac17211dfd9896a46898ad0379f606de1bf
+size 1465

checkpoint-750/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-750/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

checkpoint-750/trainer_state.json

@@ -0,0 +1,304 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.8086904043452021,
+  "eval_steps": 500,
+  "global_step": 750,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.060350030175015085,
+      "grad_norm": 0.6138771176338196,
+      "learning_rate": 0.0001894736842105263,
+      "loss": 1.3796,
+      "mean_token_accuracy": 0.7022829699516296,
+      "num_tokens": 154999.0,
+      "step": 25
+    },
+    {
+      "epoch": 0.12070006035003017,
+      "grad_norm": 0.6403661370277405,
+      "learning_rate": 0.00029993852448555923,
+      "loss": 0.7408,
+      "mean_token_accuracy": 0.8062405365705491,
+      "num_tokens": 274882.0,
+      "step": 50
+    },
+    {
+      "epoch": 0.18105009052504525,
+      "grad_norm": 0.5453419089317322,
+      "learning_rate": 0.00029934198818572623,
+      "loss": 0.4995,
+      "mean_token_accuracy": 0.8602466344833374,
+      "num_tokens": 429270.0,
+      "step": 75
+    },
+    {
+      "epoch": 0.24140012070006034,
+      "grad_norm": 0.9569323062896729,
+      "learning_rate": 0.0002981133400718627,
+      "loss": 0.3791,
+      "mean_token_accuracy": 0.8920469450950622,
+      "num_tokens": 547533.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.30175015087507545,
+      "grad_norm": 0.4823763966560364,
+      "learning_rate": 0.0002962577805768642,
+      "loss": 0.2865,
+      "mean_token_accuracy": 0.919411507844925,
+      "num_tokens": 700414.0,
+      "step": 125
+    },
+    {
+      "epoch": 0.3621001810500905,
+      "grad_norm": 0.5946052670478821,
+      "learning_rate": 0.00029378316362776546,
+      "loss": 0.3033,
+      "mean_token_accuracy": 0.9159089189767837,
+      "num_tokens": 818685.0,
+      "step": 150
+    },
+    {
+      "epoch": 0.4224502112251056,
+      "grad_norm": 0.513756513595581,
+      "learning_rate": 0.0002906999634028451,
+      "loss": 0.2095,
+      "mean_token_accuracy": 0.9410657143592834,
+      "num_tokens": 973411.0,
+      "step": 175
+    },
+    {
+      "epoch": 0.4828002414001207,
+      "grad_norm": 0.5572422742843628,
+      "learning_rate": 0.0002870212299981334,
+      "loss": 0.1986,
+      "mean_token_accuracy": 0.9448752331733704,
+      "num_tokens": 1091985.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.5431502715751357,
+      "grad_norm": 0.3195100724697113,
+      "learning_rate": 0.00028276253419097193,
+      "loss": 0.1613,
+      "mean_token_accuracy": 0.9562808114290238,
+      "num_tokens": 1249334.0,
+      "step": 225
+    },
+    {
+      "epoch": 0.6035003017501509,
+      "grad_norm": 0.8021010756492615,
+      "learning_rate": 0.00027794190153442033,
+      "loss": 0.161,
+      "mean_token_accuracy": 0.9557685399055481,
+      "num_tokens": 1367322.0,
+      "step": 250
+    },
+    {
+      "epoch": 0.663850331925166,
+      "grad_norm": 0.4245581328868866,
+      "learning_rate": 0.00027257973606146575,
+      "loss": 0.1414,
+      "mean_token_accuracy": 0.9619652438163757,
+      "num_tokens": 1523489.0,
+      "step": 275
+    },
+    {
+      "epoch": 0.724200362100181,
+      "grad_norm": 0.7518550157546997,
+      "learning_rate": 0.0002666987339219681,
+      "loss": 0.1313,
+      "mean_token_accuracy": 0.9653391098976135,
+      "num_tokens": 1644051.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.7845503922751962,
+      "grad_norm": 0.5166082978248596,
+      "learning_rate": 0.0002603237873178853,
+      "loss": 0.1179,
+      "mean_token_accuracy": 0.9685099506378174,
+      "num_tokens": 1799510.0,
+      "step": 325
+    },
+    {
+      "epoch": 0.8449004224502112,
+      "grad_norm": 0.8754676580429077,
+      "learning_rate": 0.0002534818791433866,
+      "loss": 0.1172,
+      "mean_token_accuracy": 0.9700138258934021,
+      "num_tokens": 1918554.0,
+      "step": 350
+    },
+    {
+      "epoch": 0.9052504526252263,
+      "grad_norm": 0.5072232484817505,
+      "learning_rate": 0.00024620196877580576,
+      "loss": 0.1087,
+      "mean_token_accuracy": 0.9706882643699646,
+      "num_tokens": 2071557.0,
+      "step": 375
+    },
+    {
+      "epoch": 0.9656004828002414,
+      "grad_norm": 0.44057291746139526,
+      "learning_rate": 0.00023851486950083892,
+      "loss": 0.1102,
+      "mean_token_accuracy": 0.9717801564931869,
+      "num_tokens": 2188325.0,
+      "step": 400
+    },
+    {
+      "epoch": 1.024140012070006,
+      "grad_norm": 0.28425154089927673,
+      "learning_rate": 0.00023045311809080567,
+      "loss": 0.0969,
+      "mean_token_accuracy": 0.9745202402478641,
+      "num_tokens": 2335834.0,
+      "step": 425
+    },
+    {
+      "epoch": 1.0844900422450212,
+      "grad_norm": 0.27836862206459045,
+      "learning_rate": 0.00022205083708799942,
+      "loss": 0.0775,
+      "mean_token_accuracy": 0.9795319002866745,
+      "num_tokens": 2475640.0,
+      "step": 450
+    },
+    {
+      "epoch": 1.1448400724200363,
+      "grad_norm": 0.25006264448165894,
+      "learning_rate": 0.0002133435903760353,
+      "loss": 0.0926,
+      "mean_token_accuracy": 0.9758930206298828,
+      "num_tokens": 2613115.0,
+      "step": 475
+    },
+    {
+      "epoch": 1.2051901025950513,
+      "grad_norm": 0.3176310658454895,
+      "learning_rate": 0.0002043682326505094,
+      "loss": 0.0687,
+      "mean_token_accuracy": 0.9820343238115311,
+      "num_tokens": 2749269.0,
+      "step": 500
+    },
+    {
+      "epoch": 1.2655401327700664,
+      "grad_norm": 0.20867092907428741,
+      "learning_rate": 0.000195162753426108,
+      "loss": 0.0877,
+      "mean_token_accuracy": 0.9773262399435043,
+      "num_tokens": 2883349.0,
+      "step": 525
+    },
+    {
+      "epoch": 1.3258901629450814,
+      "grad_norm": 0.25837627053260803,
+      "learning_rate": 0.00018576611624042852,
+      "loss": 0.0674,
+      "mean_token_accuracy": 0.9824212837219238,
+      "num_tokens": 3020654.0,
+      "step": 550
+    },
+    {
+      "epoch": 1.3862401931200965,
+      "grad_norm": 0.1842041015625,
+      "learning_rate": 0.00017621809373510641,
+      "loss": 0.0819,
+      "mean_token_accuracy": 0.979530062675476,
+      "num_tokens": 3157335.0,
+      "step": 575
+    },
+    {
+      "epoch": 1.4465902232951118,
+      "grad_norm": 0.2597507834434509,
+      "learning_rate": 0.00016655909931229048,
+      "loss": 0.0659,
+      "mean_token_accuracy": 0.9828319466114044,
+      "num_tokens": 3294681.0,
+      "step": 600
+    },
+    {
+      "epoch": 1.5069402534701268,
+      "grad_norm": 0.17711229622364044,
+      "learning_rate": 0.00015683001607900553,
+      "loss": 0.0842,
+      "mean_token_accuracy": 0.978652862906456,
+      "num_tokens": 3430142.0,
+      "step": 625
+    },
+    {
+      "epoch": 1.567290283645142,
+      "grad_norm": 0.13509850203990936,
+      "learning_rate": 0.00014707202380342108,
+      "loss": 0.0573,
+      "mean_token_accuracy": 0.9850119787454605,
+      "num_tokens": 3567792.0,
+      "step": 650
+    },
+    {
+      "epoch": 1.627640313820157,
+      "grad_norm": 0.14440615475177765,
+      "learning_rate": 0.00013732642461545747,
+      "loss": 0.0758,
+      "mean_token_accuracy": 0.9804488325119018,
+      "num_tokens": 3703772.0,
+      "step": 675
+    },
+    {
+      "epoch": 1.687990343995172,
+      "grad_norm": 0.12784038484096527,
+      "learning_rate": 0.00012763446818947865,
+      "loss": 0.0557,
+      "mean_token_accuracy": 0.9857299762964249,
+      "num_tokens": 3841338.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.748340374170187,
+      "grad_norm": 0.1049482449889183,
+      "learning_rate": 0.00011803717714901029,
+      "loss": 0.0692,
+      "mean_token_accuracy": 0.9823172944784164,
+      "num_tokens": 3976256.0,
+      "step": 725
+    },
+    {
+      "epoch": 1.8086904043452021,
+      "grad_norm": 0.1479799747467041,
+      "learning_rate": 0.00010857517343248423,
+      "loss": 0.0544,
+      "mean_token_accuracy": 0.9862035536766052,
+      "num_tokens": 4113650.0,
+      "step": 750
+    }
+  ],
+  "logging_steps": 25,
+  "max_steps": 1245,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 750,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.7789373371643494e+17,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-750/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1368312d6171cb3228a4505daeeebb2d50b85fd42516b8b22d1d350ef88234fe
+size 6033

runs/Aug02_22-56-09_pan/events.out.tfevents.1754189772.pan.127259.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a3cd70e873a942806a29b53e5069f8434ef71e895d0771252dfbc930c9453719
+size 22431

runs/Aug02_22-56-09_pan/events.out.tfevents.1754193848.pan.127259.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:67deb6ac5c20f246c35d8ee9601f9c6654e639534d0d887bedd09a51b2fb6e4c
+size 478

runs/Aug03_00-04-52_pan/events.out.tfevents.1754193954.pan.130463.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:df7e1a1a26e572543fc2a8c48cae25d54fd4184663305b7551ba6061b6487f9b
+size 473

special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404