Training in progress, step 375, checkpoint

.gitattributes

@@ -71,3 +71,4 @@ checkpoint-340/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 checkpoint-350/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 checkpoint-360/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 checkpoint-370/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-375/tokenizer.json filter=lfs diff=lfs merge=lfs -text

checkpoint-375/README.md

@@ -0,0 +1,209 @@
+---
+base_model: unsloth/gpt-oss-20b
+library_name: peft
+tags:
+- base_model:adapter:unsloth/gpt-oss-20b
+- lora
+- sft
+- transformers
+- trl
+- unsloth
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.0

checkpoint-375/adapter_config.json

@@ -0,0 +1,42 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "GptOssForCausalLM",
+    "parent_library": "transformers.models.gpt_oss.modeling_gpt_oss"
+  },
+  "base_model_name_or_path": "unsloth/gpt-oss-20b",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 1,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "q_proj",
+    "v_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": true
+}

checkpoint-375/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:090da726efb4a65ddc6415954434da5b4efc660a7b2300d82deb5045d26bcfce
+size 2015696

checkpoint-375/chat_template.jinja

@@ -0,0 +1,315 @@
+{# Copyright 2025-present Unsloth. Apache 2.0 License. Unsloth chat template fixes. Edited from ggml-org & OpenAI #}
+{#-
+  In addition to the normal inputs of `messages` and `tools`, this template also accepts the
+  following kwargs:
+  - "builtin_tools": A list, can contain "browser" and/or "python".
+  - "model_identity": A string that optionally describes the model identity.
+  - "reasoning_effort": A string that describes the reasoning effort, defaults to "medium".
+ #}
+
+{#- Tool Definition Rendering ============================================== #}
+{%- macro render_typescript_type(param_spec, required_params, is_nullable=false) -%}
+    {%- if param_spec.type == "array" -%}
+        {%- if param_spec['items'] -%}
+            {%- if param_spec['items']['type'] == "string" -%}
+                {{- "string[]" }}
+            {%- elif param_spec['items']['type'] == "number" -%}
+                {{- "number[]" }}
+            {%- elif param_spec['items']['type'] == "integer" -%}
+                {{- "number[]" }}
+            {%- elif param_spec['items']['type'] == "boolean" -%}
+                {{- "boolean[]" }}
+            {%- else -%}
+                {%- set inner_type = render_typescript_type(param_spec['items'], required_params) -%}
+                {%- if inner_type == "object | object" or inner_type|length > 50 -%}
+                    {{- "any[]" }}
+                {%- else -%}
+                    {{- inner_type + "[]" }}
+                {%- endif -%}
+            {%- endif -%}
+            {%- if param_spec.nullable -%}
+                {{- " | null" }}
+            {%- endif -%}
+        {%- else -%}
+            {{- "any[]" }}
+            {%- if param_spec.nullable -%}
+                {{- " | null" }}
+            {%- endif -%}
+        {%- endif -%}
+    {%- elif param_spec.type is defined and param_spec.type is iterable and param_spec.type is not string and param_spec.type is not mapping and param_spec.type[0] is defined -%}
+        {#- Handle array of types like ["object", "object"] from Union[dict, list] #}
+        {%- if param_spec.type | length > 1 -%}
+            {{- param_spec.type | join(" | ") }}
+        {%- else -%}
+            {{- param_spec.type[0] }}
+        {%- endif -%}
+    {%- elif param_spec.oneOf -%}
+        {#- Handle oneOf schemas - check for complex unions and fallback to any #}
+        {%- set has_object_variants = false -%}
+        {%- for variant in param_spec.oneOf -%}
+            {%- if variant.type == "object" -%}
+                {%- set has_object_variants = true -%}
+            {%- endif -%}
+        {%- endfor -%}
+        {%- if has_object_variants and param_spec.oneOf|length > 1 -%}
+            {{- "any" }}
+        {%- else -%}
+            {%- for variant in param_spec.oneOf -%}
+                {{- render_typescript_type(variant, required_params) -}}
+                {%- if variant.description %}
+                    {{- "// " + variant.description }}
+                {%- endif -%}
+                {%- if variant.default is defined %}
+                    {{ "// default: " + variant.default|tojson }}
+                {%- endif -%}
+                {%- if not loop.last %}
+                    {{- " | " }}
+                {% endif -%}
+            {%- endfor -%}
+        {%- endif -%}
+    {%- elif param_spec.type == "string" -%}
+        {%- if param_spec.enum -%}
+            {{- '"' + param_spec.enum|join('" | "') + '"' -}}
+        {%- else -%}
+            {{- "string" }}
+            {%- if param_spec.nullable %}
+                {{- " | null" }}
+            {%- endif -%}
+        {%- endif -%}
+    {%- elif param_spec.type == "number" -%}
+        {{- "number" }}
+    {%- elif param_spec.type == "integer" -%}
+        {{- "number" }}
+    {%- elif param_spec.type == "boolean" -%}
+        {{- "boolean" }}
+
+    {%- elif param_spec.type == "object" -%}
+        {%- if param_spec.properties -%}
+            {{- "{\n" }}
+            {%- for prop_name, prop_spec in param_spec.properties.items() -%}
+                {{- prop_name -}}
+                {%- if prop_name not in (param_spec.required or []) -%}
+                    {{- "?" }}
+                {%- endif -%}
+                {{- ": " }}
+                {{ render_typescript_type(prop_spec, param_spec.required or []) }}
+                {%- if not loop.last -%}
+                    {{-", " }}
+                {%- endif -%}
+            {%- endfor -%}
+            {{- "}" }}
+        {%- else -%}
+            {{- "object" }}
+        {%- endif -%}
+    {%- else -%}
+        {{- "any" }}
+    {%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_tool_namespace(namespace_name, tools) -%}
+    {{- "## " + namespace_name + "\n\n" }}
+    {{- "namespace " + namespace_name + " {\n\n" }}
+    {%- for tool in tools %}
+        {%- set tool = tool.function %}
+        {{- "// " + tool.description + "\n" }}
+        {{- "type "+ tool.name + " = " }}
+        {%- if tool.parameters and tool.parameters.properties -%}
+            {{- "(_: " }}
+            {{- "{\n" }}
+            {%- for param_name, param_spec in tool.parameters.properties.items() %}
+                {{- "// " + param_spec.description + "\n" }}
+                {{- param_name }}
+                {%- if param_name not in (tool.parameters.required or []) -%}
+                    {{- "?" }}
+                {%- endif -%}
+                {{- ": " }}
+                {{- render_typescript_type(param_spec, tool.parameters.required or []) }}
+                {%- if param_spec.default is defined -%}
+                    {%- if param_spec.enum %}
+                        {{- ", // default: " + param_spec.default }}
+                    {%- elif param_spec.oneOf %}
+                        {{- "// default: " + param_spec.default }}
+                    {%- else %}
+                        {{- ", // default: " + param_spec.default|tojson }}
+                    {%- endif -%}
+                {%- endif -%}
+                {%- if not loop.last %}
+                    {{- ",\n" }}
+                {%- else %}
+                    {{- "\n" }}
+                {%- endif -%}
+            {%- endfor %}
+            {{- "}) => any;\n\n" }}
+        {%- else -%}
+            {{- "() => any;\n\n" }}
+        {%- endif -%}
+    {%- endfor %}
+    {{- "} // namespace " + namespace_name }}
+{%- endmacro -%}
+
+{%- macro render_builtin_tools(browser_tool, python_tool) -%}
+    {%- if browser_tool %}
+        {{- "## browser\n\n" }}
+        {{- "// Tool for browsing.\n" }}
+        {{- "// The `cursor` appears in brackets before each browsing display: `[{cursor}]`.\n" }}
+        {{- "// Cite information from the tool using the following format:\n" }}
+        {{- "// `【{cursor}†L{line_start}(-L{line_end})?】`, for example: `【6†L9-L11】` or `【8†L3】`.\n" }}
+        {{- "// Do not quote more than 10 words directly from the tool output.\n" }}
+        {{- "// sources=web (default: web)\n" }}
+        {{- "namespace browser {\n\n" }}
+        {{- "// Searches for information related to `query` and displays `topn` results.\n" }}
+        {{- "type search = (_: {\n" }}
+        {{- "query: string,\n" }}
+        {{- "topn?: number, // default: 10\n" }}
+        {{- "source?: string,\n" }}
+        {{- "}) => any;\n\n" }}
+        {{- "// Opens the link `id` from the page indicated by `cursor` starting at line number `loc`, showing `num_lines` lines.\n" }}
+        {{- "// Valid link ids are displayed with the formatting: `【{id}†.*】`.\n" }}
+        {{- "// If `cursor` is not provided, the most recent page is implied.\n" }}
+        {{- "// If `id` is a string, it is treated as a fully qualified URL associated with `source`.\n" }}
+        {{- "// If `loc` is not provided, the viewport will be positioned at the beginning of the document or centered on the most relevant passage, if available.\n" }}
+        {{- "// Use this function without `id` to scroll to a new location of an opened page.\n" }}
+        {{- "type open = (_: {\n" }}
+        {{- "id?: number | string, // default: -1\n" }}
+        {{- "cursor?: number, // default: -1\n" }}
+        {{- "loc?: number, // default: -1\n" }}
+        {{- "num_lines?: number, // default: -1\n" }}
+        {{- "view_source?: boolean, // default: false\n" }}
+        {{- "source?: string,\n" }}
+        {{- "}) => any;\n\n" }}
+        {{- "// Finds exact matches of `pattern` in the current page, or the page given by `cursor`.\n" }}
+        {{- "type find = (_: {\n" }}
+        {{- "pattern: string,\n" }}
+        {{- "cursor?: number, // default: -1\n" }}
+        {{- "}) => any;\n\n" }}
+        {{- "} // namespace browser\n\n" }}
+    {%- endif -%}
+
+    {%- if python_tool %}
+        {{- "## python\n\n" }}
+        {{- "Use this tool to execute Python code in your chain of thought. The code will not be shown to the user. This tool should be used for internal reasoning, but not for code that is intended to be visible to the user (e.g. when creating plots, tables, or files).\n\n" }}
+        {{- "When you send a message containing Python code to python, it will be executed in a stateful Jupyter notebook environment. python will respond with the output of the execution or time out after 120.0 seconds. The drive at '/mnt/data' can be used to save and persist user files. Internet access for this session is UNKNOWN. Depends on the cluster.\n\n" }}
+    {%- endif -%}
+{%- endmacro -%}
+
+{#- System Message Construction ============================================ #}
+{%- macro build_system_message() -%}
+    {%- if model_identity is not defined %}
+        {{- "You are ChatGPT, a large language model trained by OpenAI.\n" -}}
+    {%- else %}
+        {{- model_identity }}
+    {%- endif %}
+    {{- "Knowledge cutoff: 2024-06\n" }}
+    {{- "Current date: " + strftime_now("%Y-%m-%d") + "\n\n" }}
+    {%- if reasoning_effort is not defined %}
+        {%- set reasoning_effort = "medium" %}
+    {%- endif %}
+    {{- "Reasoning: " + reasoning_effort + "\n\n" }}
+    {%- if builtin_tools is defined %}
+        {{- "# Tools\n\n" }}
+        {%- set available_builtin_tools = namespace(browser=false, python=false) %}
+        {%- for tool in builtin_tools %}
+            {%- if tool == "browser" %}
+                {%- set available_builtin_tools.browser = true %}
+            {%- elif tool == "python" %}
+                {%- set available_builtin_tools.python = true %}
+            {%- endif %}
+        {%- endfor %}
+        {{- render_builtin_tools(available_builtin_tools.browser, available_builtin_tools.python) }}
+    {%- endif -%}
+    {{- "# Valid channels: analysis, commentary, final. Channel must be included for every message." }}
+    {%- if tools is defined -%}
+        {{- "\nCalls to these tools must go to the commentary channel: 'functions'." }}
+    {%- endif -%}
+{%- endmacro -%}
+
+{#- Main Template Logic ================================================= #}
+{#- Set defaults #}
+
+{#- Render system message #}
+{{- "<|start|>system<|message|>" }}
+{{- build_system_message() }}
+{{- "<|end|>" }}
+
+{#- Extract developer message #}
+{%- if messages[0].role == "developer" or messages[0].role == "system" %}
+    {%- set developer_message = messages[0].content %}
+    {%- set loop_messages = messages[1:] %}
+{%- else %}
+    {%- set developer_message = "" %}
+    {%- set loop_messages = messages %}
+{%- endif %}
+
+{#- Render developer message #}
+{%- if developer_message or tools %}
+    {{- "<|start|>developer<|message|>" }}
+    {%- if developer_message %}
+        {{- "# Instructions\n\n" }}
+        {{- developer_message }}
+    {%- endif %}
+    {%- if tools -%}
+        {{- "\n\n" }}
+        {{- "# Tools\n\n" }}
+        {{- render_tool_namespace("functions", tools) }}
+    {%- endif -%}
+    {{- "<|end|>" }}
+{%- endif %}
+
+{#- Render messages #}
+{%- set last_tool_call = namespace(name=none) %}
+{%- for message in loop_messages -%}
+    {#- At this point only assistant/user/tool messages should remain #}
+    {%- if message.role == 'assistant' -%}
+        {%- if "tool_calls" in message %}
+            {#- We assume max 1 tool call per message, and so we infer the tool call name #}
+            {#- in "tool" messages from the most recent assistant tool call name #}
+            {%- set tool_call = message.tool_calls[0] %}
+            {%- if tool_call.function %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {%- if message.content %}
+                {{- "<|start|>assistant<|channel|>analysis<|message|>" + message.content + "<|end|>" }}
+            {%- endif %}
+            {{- "<|start|>assistant to=" }}
+            {{- "functions." + tool_call.name + "<|channel|>commentary json<|message|>" }}
+            {{- tool_call.arguments|tojson }}
+            {{- "<|call|>" }}
+            {%- set last_tool_call.name = tool_call.name %}
+        {%- elif "thinking" in message and loop.last and not add_generation_prompt %}
+            {#- Only render the CoT if the final turn is an assistant turn and add_generation_prompt is false #}
+            {#- This is a situation that should only occur in training, never in inference. #}
+            {{- "<|start|>assistant<|channel|>analysis<|message|>" + message.thinking + "<|end|>" }}
+            {#- <|return|> indicates the end of generation, but <|end|> does not #}
+            {#- <|return|> should never be an input to the model, but we include it as the final token #}
+            {#- when training, so the model learns to emit it. #}
+            {{- "<|start|>assistant<|channel|>final<|message|>" + message.content + "<|return|>" }}
+            {%- set last_tool_call.name = none %}
+        {%- elif "thinking" in message %}
+            {#- CoT is dropped during all previous turns, so we never render it for inference #}
+            {{- "<|start|>assistant<|channel|>final<|message|>" + message.content + "<|end|>" }}
+            {%- set last_tool_call.name = none %}
+        {%- elif loop.last and not add_generation_prompt %}
+            {#- <|return|> indicates the end of generation, but <|end|> does not #}
+            {#- <|return|> should never be an input to the model, but we include it as the final token #}
+            {#- when training, so the model learns to emit it. #}
+            {{- "<|start|>assistant<|message|>" + message.content + "<|return|>" }}
+        {%- else %}
+            {{- "<|start|>assistant<|message|>" + message.content + "<|end|>" }}
+            {%- set last_tool_call.name = none %}
+        {%- endif %}
+    {%- elif message.role == 'tool' -%}
+        {%- if last_tool_call.name is none %}
+            {{- raise_exception("Message has tool role, but there was no previous assistant message with a tool call!") }}
+        {%- endif %}
+        {{- "<|start|>functions." + last_tool_call.name }}
+        {{- " to=assistant<|channel|>commentary<|message|>" + message.content|tojson + "<|end|>" }}
+    {%- else -%}
+        {{- "<|start|>user<|message|>" + message.content + "<|end|>" }}
+    {%- endif -%}
+{%- endfor -%}
+
+{#- Generation prompt #}
+{%- if add_generation_prompt -%}
+<|start|>assistant
+{%- endif -%}
+{# Copyright 2025-present Unsloth. Apache 2.0 License. Unsloth chat template fixes. Edited from ggml-org & OpenAI #}

checkpoint-375/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cf597f2773a5e243d68778cb02abb0754f14b9c6958b8553a41c9179bedc81e3
+size 2951883

checkpoint-375/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c226d31b8a31330f2ee850b10bd7ac149a57aff7ded0041721f81319ff75e9b
+size 14581

checkpoint-375/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e218fb5a057c8bad3409875137fd669abfbb61a30b591edea9a460e390fbcd1c
+size 1465

checkpoint-375/special_tokens_map.json

@@ -0,0 +1,23 @@
+{
+  "bos_token": {
+    "content": "<|startoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|return|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|reserved_200017|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-375/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0614fe83cadab421296e664e1f48f4261fa8fef6e03e63bb75c20f38e37d07d3
+size 27868174

checkpoint-375/tokenizer_config.json

@@ -0,0 +1,185 @@
+{
+  "added_tokens_decoder": {
+    "199998": {
+      "content": "<|startoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "199999": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "200000": {
+      "content": "<|reserved_200000|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "200001": {
+      "content": "<|reserved_200001|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "200002": {
+      "content": "<|return|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "200003": {
+      "content": "<|constrain|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "200004": {
+      "content": "<|reserved_200004|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "200005": {
+      "content": "<|channel|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "200006": {
+      "content": "<|start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "200007": {
+      "content": "<|end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "200008": {
+      "content": "<|message|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "200009": {
+      "content": "<|reserved_200009|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "200010": {
+      "content": "<|reserved_200010|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "200011": {
+      "content": "<|reserved_200011|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "200012": {
+      "content": "<|call|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "200013": {
+      "content": "<|reserved_200013|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "200014": {
+      "content": "<|reserved_200014|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "200015": {
+      "content": "<|reserved_200015|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "200016": {
+      "content": "<|reserved_200016|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "200017": {
+      "content": "<|reserved_200017|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "200018": {
+      "content": "<|endofprompt|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|startoftext|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|return|>",
+  "extra_special_tokens": {},
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 131072,
+  "pad_token": "<|reserved_200017|>",
+  "padding_side": "right",
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "unk_token": null
+}

checkpoint-375/trainer_state.json

@@ -0,0 +1,2955 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.0,
+  "eval_steps": 10,
+  "global_step": 375,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0026666666666666666,
+      "grad_norm": 932.6458129882812,
+      "learning_rate": 0.0,
+      "loss": 16.4322,
+      "step": 1
+    },
+    {
+      "epoch": 0.005333333333333333,
+      "grad_norm": 906.1286010742188,
+      "learning_rate": 2.0000000000000003e-06,
+      "loss": 17.1074,
+      "step": 2
+    },
+    {
+      "epoch": 0.008,
+      "grad_norm": 957.1098022460938,
+      "learning_rate": 4.000000000000001e-06,
+      "loss": 16.2636,
+      "step": 3
+    },
+    {
+      "epoch": 0.010666666666666666,
+      "grad_norm": 999.4998779296875,
+      "learning_rate": 6e-06,
+      "loss": 16.7171,
+      "step": 4
+    },
+    {
+      "epoch": 0.013333333333333334,
+      "grad_norm": 952.225341796875,
+      "learning_rate": 8.000000000000001e-06,
+      "loss": 16.0368,
+      "step": 5
+    },
+    {
+      "epoch": 0.016,
+      "grad_norm": 967.4716186523438,
+      "learning_rate": 1e-05,
+      "loss": 15.3585,
+      "step": 6
+    },
+    {
+      "epoch": 0.018666666666666668,
+      "grad_norm": 1014.7380981445312,
+      "learning_rate": 9.972972972972975e-06,
+      "loss": 14.1371,
+      "step": 7
+    },
+    {
+      "epoch": 0.021333333333333333,
+      "grad_norm": 1001.39404296875,
+      "learning_rate": 9.945945945945947e-06,
+      "loss": 12.4417,
+      "step": 8
+    },
+    {
+      "epoch": 0.024,
+      "grad_norm": 956.663818359375,
+      "learning_rate": 9.91891891891892e-06,
+      "loss": 11.2694,
+      "step": 9
+    },
+    {
+      "epoch": 0.02666666666666667,
+      "grad_norm": 851.0857543945312,
+      "learning_rate": 9.891891891891893e-06,
+      "loss": 10.1807,
+      "step": 10
+    },
+    {
+      "epoch": 0.02666666666666667,
+      "eval_loss": 8.908344268798828,
+      "eval_runtime": 6.3541,
+      "eval_samples_per_second": 4.721,
+      "eval_steps_per_second": 2.361,
+      "step": 10
+    },
+    {
+      "epoch": 0.029333333333333333,
+      "grad_norm": 689.436767578125,
+      "learning_rate": 9.864864864864865e-06,
+      "loss": 8.85,
+      "step": 11
+    },
+    {
+      "epoch": 0.032,
+      "grad_norm": 591.1151123046875,
+      "learning_rate": 9.83783783783784e-06,
+      "loss": 8.4188,
+      "step": 12
+    },
+    {
+      "epoch": 0.034666666666666665,
+      "grad_norm": 414.6604919433594,
+      "learning_rate": 9.810810810810811e-06,
+      "loss": 7.1033,
+      "step": 13
+    },
+    {
+      "epoch": 0.037333333333333336,
+      "grad_norm": 339.56982421875,
+      "learning_rate": 9.783783783783785e-06,
+      "loss": 7.1921,
+      "step": 14
+    },
+    {
+      "epoch": 0.04,
+      "grad_norm": 304.45819091796875,
+      "learning_rate": 9.756756756756758e-06,
+      "loss": 6.5019,
+      "step": 15
+    },
+    {
+      "epoch": 0.042666666666666665,
+      "grad_norm": 244.03823852539062,
+      "learning_rate": 9.729729729729732e-06,
+      "loss": 6.4202,
+      "step": 16
+    },
+    {
+      "epoch": 0.04533333333333334,
+      "grad_norm": 204.49368286132812,
+      "learning_rate": 9.702702702702704e-06,
+      "loss": 5.91,
+      "step": 17
+    },
+    {
+      "epoch": 0.048,
+      "grad_norm": 162.0959014892578,
+      "learning_rate": 9.675675675675676e-06,
+      "loss": 5.9435,
+      "step": 18
+    },
+    {
+      "epoch": 0.050666666666666665,
+      "grad_norm": 160.5931396484375,
+      "learning_rate": 9.64864864864865e-06,
+      "loss": 5.6556,
+      "step": 19
+    },
+    {
+      "epoch": 0.05333333333333334,
+      "grad_norm": 146.50758361816406,
+      "learning_rate": 9.621621621621622e-06,
+      "loss": 5.459,
+      "step": 20
+    },
+    {
+      "epoch": 0.05333333333333334,
+      "eval_loss": 5.419591426849365,
+      "eval_runtime": 2.1235,
+      "eval_samples_per_second": 14.127,
+      "eval_steps_per_second": 7.064,
+      "step": 20
+    },
+    {
+      "epoch": 0.056,
+      "grad_norm": 131.01402282714844,
+      "learning_rate": 9.594594594594594e-06,
+      "loss": 5.5075,
+      "step": 21
+    },
+    {
+      "epoch": 0.058666666666666666,
+      "grad_norm": 130.70143127441406,
+      "learning_rate": 9.567567567567568e-06,
+      "loss": 5.2305,
+      "step": 22
+    },
+    {
+      "epoch": 0.06133333333333333,
+      "grad_norm": 118.18517303466797,
+      "learning_rate": 9.540540540540542e-06,
+      "loss": 5.0906,
+      "step": 23
+    },
+    {
+      "epoch": 0.064,
+      "grad_norm": 117.36890411376953,
+      "learning_rate": 9.513513513513514e-06,
+      "loss": 4.939,
+      "step": 24
+    },
+    {
+      "epoch": 0.06666666666666667,
+      "grad_norm": 116.87814331054688,
+      "learning_rate": 9.486486486486487e-06,
+      "loss": 4.7326,
+      "step": 25
+    },
+    {
+      "epoch": 0.06933333333333333,
+      "grad_norm": 113.54383850097656,
+      "learning_rate": 9.45945945945946e-06,
+      "loss": 4.812,
+      "step": 26
+    },
+    {
+      "epoch": 0.072,
+      "grad_norm": 104.74027252197266,
+      "learning_rate": 9.432432432432433e-06,
+      "loss": 4.6992,
+      "step": 27
+    },
+    {
+      "epoch": 0.07466666666666667,
+      "grad_norm": 93.894287109375,
+      "learning_rate": 9.405405405405407e-06,
+      "loss": 4.2069,
+      "step": 28
+    },
+    {
+      "epoch": 0.07733333333333334,
+      "grad_norm": 97.65583801269531,
+      "learning_rate": 9.378378378378379e-06,
+      "loss": 4.323,
+      "step": 29
+    },
+    {
+      "epoch": 0.08,
+      "grad_norm": 96.1834487915039,
+      "learning_rate": 9.351351351351353e-06,
+      "loss": 4.2722,
+      "step": 30
+    },
+    {
+      "epoch": 0.08,
+      "eval_loss": 4.122442245483398,
+      "eval_runtime": 2.1149,
+      "eval_samples_per_second": 14.185,
+      "eval_steps_per_second": 7.092,
+      "step": 30
+    },
+    {
+      "epoch": 0.08266666666666667,
+      "grad_norm": 85.94340515136719,
+      "learning_rate": 9.324324324324325e-06,
+      "loss": 4.0937,
+      "step": 31
+    },
+    {
+      "epoch": 0.08533333333333333,
+      "grad_norm": 83.52753448486328,
+      "learning_rate": 9.297297297297299e-06,
+      "loss": 3.917,
+      "step": 32
+    },
+    {
+      "epoch": 0.088,
+      "grad_norm": 78.75122833251953,
+      "learning_rate": 9.270270270270271e-06,
+      "loss": 3.7462,
+      "step": 33
+    },
+    {
+      "epoch": 0.09066666666666667,
+      "grad_norm": 78.64039611816406,
+      "learning_rate": 9.243243243243243e-06,
+      "loss": 3.8485,
+      "step": 34
+    },
+    {
+      "epoch": 0.09333333333333334,
+      "grad_norm": 76.619384765625,
+      "learning_rate": 9.216216216216217e-06,
+      "loss": 3.8375,
+      "step": 35
+    },
+    {
+      "epoch": 0.096,
+      "grad_norm": 67.17459869384766,
+      "learning_rate": 9.189189189189191e-06,
+      "loss": 3.6146,
+      "step": 36
+    },
+    {
+      "epoch": 0.09866666666666667,
+      "grad_norm": 77.51924896240234,
+      "learning_rate": 9.162162162162162e-06,
+      "loss": 3.5797,
+      "step": 37
+    },
+    {
+      "epoch": 0.10133333333333333,
+      "grad_norm": 65.51860046386719,
+      "learning_rate": 9.135135135135136e-06,
+      "loss": 3.4923,
+      "step": 38
+    },
+    {
+      "epoch": 0.104,
+      "grad_norm": 59.56484603881836,
+      "learning_rate": 9.10810810810811e-06,
+      "loss": 3.3492,
+      "step": 39
+    },
+    {
+      "epoch": 0.10666666666666667,
+      "grad_norm": 61.95277404785156,
+      "learning_rate": 9.081081081081082e-06,
+      "loss": 3.208,
+      "step": 40
+    },
+    {
+      "epoch": 0.10666666666666667,
+      "eval_loss": 3.2251017093658447,
+      "eval_runtime": 2.108,
+      "eval_samples_per_second": 14.231,
+      "eval_steps_per_second": 7.116,
+      "step": 40
+    },
+    {
+      "epoch": 0.10933333333333334,
+      "grad_norm": 58.99920654296875,
+      "learning_rate": 9.054054054054054e-06,
+      "loss": 3.0812,
+      "step": 41
+    },
+    {
+      "epoch": 0.112,
+      "grad_norm": 51.660400390625,
+      "learning_rate": 9.027027027027028e-06,
+      "loss": 3.0894,
+      "step": 42
+    },
+    {
+      "epoch": 0.11466666666666667,
+      "grad_norm": 41.950157165527344,
+      "learning_rate": 9e-06,
+      "loss": 3.0536,
+      "step": 43
+    },
+    {
+      "epoch": 0.11733333333333333,
+      "grad_norm": 44.57719421386719,
+      "learning_rate": 8.972972972972974e-06,
+      "loss": 3.1747,
+      "step": 44
+    },
+    {
+      "epoch": 0.12,
+      "grad_norm": 44.626502990722656,
+      "learning_rate": 8.945945945945946e-06,
+      "loss": 2.9056,
+      "step": 45
+    },
+    {
+      "epoch": 0.12266666666666666,
+      "grad_norm": 46.089561462402344,
+      "learning_rate": 8.91891891891892e-06,
+      "loss": 2.8722,
+      "step": 46
+    },
+    {
+      "epoch": 0.12533333333333332,
+      "grad_norm": 39.88739013671875,
+      "learning_rate": 8.891891891891893e-06,
+      "loss": 2.7787,
+      "step": 47
+    },
+    {
+      "epoch": 0.128,
+      "grad_norm": 48.06010055541992,
+      "learning_rate": 8.864864864864866e-06,
+      "loss": 2.6416,
+      "step": 48
+    },
+    {
+      "epoch": 0.13066666666666665,
+      "grad_norm": 43.671119689941406,
+      "learning_rate": 8.837837837837839e-06,
+      "loss": 2.7215,
+      "step": 49
+    },
+    {
+      "epoch": 0.13333333333333333,
+      "grad_norm": 51.33174514770508,
+      "learning_rate": 8.810810810810811e-06,
+      "loss": 2.7913,
+      "step": 50
+    },
+    {
+      "epoch": 0.13333333333333333,
+      "eval_loss": 2.6058502197265625,
+      "eval_runtime": 2.1101,
+      "eval_samples_per_second": 14.217,
+      "eval_steps_per_second": 7.109,
+      "step": 50
+    },
+    {
+      "epoch": 0.136,
+      "grad_norm": 34.640682220458984,
+      "learning_rate": 8.783783783783785e-06,
+      "loss": 2.554,
+      "step": 51
+    },
+    {
+      "epoch": 0.13866666666666666,
+      "grad_norm": 32.68441390991211,
+      "learning_rate": 8.756756756756759e-06,
+      "loss": 2.3897,
+      "step": 52
+    },
+    {
+      "epoch": 0.14133333333333334,
+      "grad_norm": 34.406028747558594,
+      "learning_rate": 8.72972972972973e-06,
+      "loss": 2.4736,
+      "step": 53
+    },
+    {
+      "epoch": 0.144,
+      "grad_norm": 38.37754821777344,
+      "learning_rate": 8.702702702702703e-06,
+      "loss": 2.4567,
+      "step": 54
+    },
+    {
+      "epoch": 0.14666666666666667,
+      "grad_norm": 37.39752197265625,
+      "learning_rate": 8.675675675675677e-06,
+      "loss": 2.5476,
+      "step": 55
+    },
+    {
+      "epoch": 0.14933333333333335,
+      "grad_norm": 32.165985107421875,
+      "learning_rate": 8.64864864864865e-06,
+      "loss": 2.3649,
+      "step": 56
+    },
+    {
+      "epoch": 0.152,
+      "grad_norm": 37.88535690307617,
+      "learning_rate": 8.621621621621622e-06,
+      "loss": 2.4366,
+      "step": 57
+    },
+    {
+      "epoch": 0.15466666666666667,
+      "grad_norm": 44.71893310546875,
+      "learning_rate": 8.594594594594595e-06,
+      "loss": 2.3843,
+      "step": 58
+    },
+    {
+      "epoch": 0.15733333333333333,
+      "grad_norm": 50.348106384277344,
+      "learning_rate": 8.567567567567568e-06,
+      "loss": 2.3169,
+      "step": 59
+    },
+    {
+      "epoch": 0.16,
+      "grad_norm": 39.178260803222656,
+      "learning_rate": 8.540540540540542e-06,
+      "loss": 2.15,
+      "step": 60
+    },
+    {
+      "epoch": 0.16,
+      "eval_loss": 2.204402208328247,
+      "eval_runtime": 1.4436,
+      "eval_samples_per_second": 20.781,
+      "eval_steps_per_second": 10.39,
+      "step": 60
+    },
+    {
+      "epoch": 0.16266666666666665,
+      "grad_norm": 31.640228271484375,
+      "learning_rate": 8.513513513513514e-06,
+      "loss": 2.292,
+      "step": 61
+    },
+    {
+      "epoch": 0.16533333333333333,
+      "grad_norm": 25.845205307006836,
+      "learning_rate": 8.486486486486488e-06,
+      "loss": 2.1367,
+      "step": 62
+    },
+    {
+      "epoch": 0.168,
+      "grad_norm": 24.244155883789062,
+      "learning_rate": 8.45945945945946e-06,
+      "loss": 2.2619,
+      "step": 63
+    },
+    {
+      "epoch": 0.17066666666666666,
+      "grad_norm": 27.788658142089844,
+      "learning_rate": 8.432432432432434e-06,
+      "loss": 2.0973,
+      "step": 64
+    },
+    {
+      "epoch": 0.17333333333333334,
+      "grad_norm": 22.125831604003906,
+      "learning_rate": 8.405405405405406e-06,
+      "loss": 2.0382,
+      "step": 65
+    },
+    {
+      "epoch": 0.176,
+      "grad_norm": 21.390626907348633,
+      "learning_rate": 8.378378378378378e-06,
+      "loss": 2.1409,
+      "step": 66
+    },
+    {
+      "epoch": 0.17866666666666667,
+      "grad_norm": 21.641523361206055,
+      "learning_rate": 8.351351351351352e-06,
+      "loss": 2.0422,
+      "step": 67
+    },
+    {
+      "epoch": 0.18133333333333335,
+      "grad_norm": 20.90644073486328,
+      "learning_rate": 8.324324324324326e-06,
+      "loss": 2.1016,
+      "step": 68
+    },
+    {
+      "epoch": 0.184,
+      "grad_norm": 23.33363151550293,
+      "learning_rate": 8.297297297297298e-06,
+      "loss": 2.1136,
+      "step": 69
+    },
+    {
+      "epoch": 0.18666666666666668,
+      "grad_norm": 18.258081436157227,
+      "learning_rate": 8.27027027027027e-06,
+      "loss": 1.973,
+      "step": 70
+    },
+    {
+      "epoch": 0.18666666666666668,
+      "eval_loss": 1.9927349090576172,
+      "eval_runtime": 1.4579,
+      "eval_samples_per_second": 20.577,
+      "eval_steps_per_second": 10.289,
+      "step": 70
+    },
+    {
+      "epoch": 0.18933333333333333,
+      "grad_norm": 18.579404830932617,
+      "learning_rate": 8.243243243243245e-06,
+      "loss": 2.0731,
+      "step": 71
+    },
+    {
+      "epoch": 0.192,
+      "grad_norm": 17.585365295410156,
+      "learning_rate": 8.216216216216217e-06,
+      "loss": 1.9039,
+      "step": 72
+    },
+    {
+      "epoch": 0.19466666666666665,
+      "grad_norm": 16.354633331298828,
+      "learning_rate": 8.189189189189189e-06,
+      "loss": 2.0146,
+      "step": 73
+    },
+    {
+      "epoch": 0.19733333333333333,
+      "grad_norm": 17.63299560546875,
+      "learning_rate": 8.162162162162163e-06,
+      "loss": 1.8823,
+      "step": 74
+    },
+    {
+      "epoch": 0.2,
+      "grad_norm": 16.70092010498047,
+      "learning_rate": 8.135135135135137e-06,
+      "loss": 1.8186,
+      "step": 75
+    },
+    {
+      "epoch": 0.20266666666666666,
+      "grad_norm": 16.694368362426758,
+      "learning_rate": 8.108108108108109e-06,
+      "loss": 1.8642,
+      "step": 76
+    },
+    {
+      "epoch": 0.20533333333333334,
+      "grad_norm": 16.945659637451172,
+      "learning_rate": 8.081081081081081e-06,
+      "loss": 1.832,
+      "step": 77
+    },
+    {
+      "epoch": 0.208,
+      "grad_norm": 14.920422554016113,
+      "learning_rate": 8.054054054054055e-06,
+      "loss": 1.8666,
+      "step": 78
+    },
+    {
+      "epoch": 0.21066666666666667,
+      "grad_norm": 15.18919563293457,
+      "learning_rate": 8.027027027027027e-06,
+      "loss": 1.8289,
+      "step": 79
+    },
+    {
+      "epoch": 0.21333333333333335,
+      "grad_norm": 16.04713249206543,
+      "learning_rate": 8.000000000000001e-06,
+      "loss": 1.8633,
+      "step": 80
+    },
+    {
+      "epoch": 0.21333333333333335,
+      "eval_loss": 1.8755805492401123,
+      "eval_runtime": 1.4512,
+      "eval_samples_per_second": 20.673,
+      "eval_steps_per_second": 10.336,
+      "step": 80
+    },
+    {
+      "epoch": 0.216,
+      "grad_norm": 17.620529174804688,
+      "learning_rate": 7.972972972972974e-06,
+      "loss": 1.9985,
+      "step": 81
+    },
+    {
+      "epoch": 0.21866666666666668,
+      "grad_norm": 13.032703399658203,
+      "learning_rate": 7.945945945945946e-06,
+      "loss": 1.8214,
+      "step": 82
+    },
+    {
+      "epoch": 0.22133333333333333,
+      "grad_norm": 14.5070219039917,
+      "learning_rate": 7.91891891891892e-06,
+      "loss": 1.7842,
+      "step": 83
+    },
+    {
+      "epoch": 0.224,
+      "grad_norm": 15.286811828613281,
+      "learning_rate": 7.891891891891894e-06,
+      "loss": 1.9336,
+      "step": 84
+    },
+    {
+      "epoch": 0.22666666666666666,
+      "grad_norm": 14.387998580932617,
+      "learning_rate": 7.864864864864866e-06,
+      "loss": 1.9154,
+      "step": 85
+    },
+    {
+      "epoch": 0.22933333333333333,
+      "grad_norm": 14.496636390686035,
+      "learning_rate": 7.837837837837838e-06,
+      "loss": 1.7855,
+      "step": 86
+    },
+    {
+      "epoch": 0.232,
+      "grad_norm": 13.50484848022461,
+      "learning_rate": 7.810810810810812e-06,
+      "loss": 1.782,
+      "step": 87
+    },
+    {
+      "epoch": 0.23466666666666666,
+      "grad_norm": 16.108259201049805,
+      "learning_rate": 7.783783783783784e-06,
+      "loss": 1.986,
+      "step": 88
+    },
+    {
+      "epoch": 0.23733333333333334,
+      "grad_norm": 14.42662525177002,
+      "learning_rate": 7.756756756756756e-06,
+      "loss": 1.7175,
+      "step": 89
+    },
+    {
+      "epoch": 0.24,
+      "grad_norm": 12.877999305725098,
+      "learning_rate": 7.72972972972973e-06,
+      "loss": 1.8143,
+      "step": 90
+    },
+    {
+      "epoch": 0.24,
+      "eval_loss": 1.7911643981933594,
+      "eval_runtime": 1.4513,
+      "eval_samples_per_second": 20.671,
+      "eval_steps_per_second": 10.335,
+      "step": 90
+    },
+    {
+      "epoch": 0.24266666666666667,
+      "grad_norm": 14.032076835632324,
+      "learning_rate": 7.702702702702704e-06,
+      "loss": 1.8842,
+      "step": 91
+    },
+    {
+      "epoch": 0.24533333333333332,
+      "grad_norm": 14.625088691711426,
+      "learning_rate": 7.675675675675676e-06,
+      "loss": 1.8891,
+      "step": 92
+    },
+    {
+      "epoch": 0.248,
+      "grad_norm": 12.943449020385742,
+      "learning_rate": 7.648648648648649e-06,
+      "loss": 1.69,
+      "step": 93
+    },
+    {
+      "epoch": 0.25066666666666665,
+      "grad_norm": 14.895347595214844,
+      "learning_rate": 7.621621621621622e-06,
+      "loss": 1.8021,
+      "step": 94
+    },
+    {
+      "epoch": 0.25333333333333335,
+      "grad_norm": 14.53865909576416,
+      "learning_rate": 7.594594594594596e-06,
+      "loss": 1.7205,
+      "step": 95
+    },
+    {
+      "epoch": 0.256,
+      "grad_norm": 12.052194595336914,
+      "learning_rate": 7.567567567567569e-06,
+      "loss": 1.7664,
+      "step": 96
+    },
+    {
+      "epoch": 0.25866666666666666,
+      "grad_norm": 12.567586898803711,
+      "learning_rate": 7.540540540540541e-06,
+      "loss": 1.7944,
+      "step": 97
+    },
+    {
+      "epoch": 0.2613333333333333,
+      "grad_norm": 14.633405685424805,
+      "learning_rate": 7.513513513513514e-06,
+      "loss": 1.6582,
+      "step": 98
+    },
+    {
+      "epoch": 0.264,
+      "grad_norm": 13.399588584899902,
+      "learning_rate": 7.486486486486487e-06,
+      "loss": 1.8577,
+      "step": 99
+    },
+    {
+      "epoch": 0.26666666666666666,
+      "grad_norm": 12.297356605529785,
+      "learning_rate": 7.45945945945946e-06,
+      "loss": 1.817,
+      "step": 100
+    },
+    {
+      "epoch": 0.26666666666666666,
+      "eval_loss": 1.7191141843795776,
+      "eval_runtime": 1.4464,
+      "eval_samples_per_second": 20.741,
+      "eval_steps_per_second": 10.37,
+      "step": 100
+    },
+    {
+      "epoch": 0.2693333333333333,
+      "grad_norm": 13.96349048614502,
+      "learning_rate": 7.4324324324324324e-06,
+      "loss": 1.7299,
+      "step": 101
+    },
+    {
+      "epoch": 0.272,
+      "grad_norm": 12.001126289367676,
+      "learning_rate": 7.4054054054054055e-06,
+      "loss": 1.6544,
+      "step": 102
+    },
+    {
+      "epoch": 0.27466666666666667,
+      "grad_norm": 13.116267204284668,
+      "learning_rate": 7.3783783783783794e-06,
+      "loss": 1.7013,
+      "step": 103
+    },
+    {
+      "epoch": 0.2773333333333333,
+      "grad_norm": 13.404566764831543,
+      "learning_rate": 7.3513513513513525e-06,
+      "loss": 1.7313,
+      "step": 104
+    },
+    {
+      "epoch": 0.28,
+      "grad_norm": 13.112600326538086,
+      "learning_rate": 7.324324324324325e-06,
+      "loss": 1.6768,
+      "step": 105
+    },
+    {
+      "epoch": 0.2826666666666667,
+      "grad_norm": 11.644760131835938,
+      "learning_rate": 7.297297297297298e-06,
+      "loss": 1.7202,
+      "step": 106
+    },
+    {
+      "epoch": 0.2853333333333333,
+      "grad_norm": 12.4971923828125,
+      "learning_rate": 7.270270270270271e-06,
+      "loss": 1.6554,
+      "step": 107
+    },
+    {
+      "epoch": 0.288,
+      "grad_norm": 13.368678092956543,
+      "learning_rate": 7.243243243243244e-06,
+      "loss": 1.7155,
+      "step": 108
+    },
+    {
+      "epoch": 0.2906666666666667,
+      "grad_norm": 11.646718978881836,
+      "learning_rate": 7.216216216216216e-06,
+      "loss": 1.6875,
+      "step": 109
+    },
+    {
+      "epoch": 0.29333333333333333,
+      "grad_norm": 11.237492561340332,
+      "learning_rate": 7.189189189189189e-06,
+      "loss": 1.7969,
+      "step": 110
+    },
+    {
+      "epoch": 0.29333333333333333,
+      "eval_loss": 1.6631710529327393,
+      "eval_runtime": 1.4481,
+      "eval_samples_per_second": 20.717,
+      "eval_steps_per_second": 10.358,
+      "step": 110
+    },
+    {
+      "epoch": 0.296,
+      "grad_norm": 12.128864288330078,
+      "learning_rate": 7.162162162162163e-06,
+      "loss": 1.71,
+      "step": 111
+    },
+    {
+      "epoch": 0.2986666666666667,
+      "grad_norm": 13.497359275817871,
+      "learning_rate": 7.135135135135136e-06,
+      "loss": 1.6549,
+      "step": 112
+    },
+    {
+      "epoch": 0.30133333333333334,
+      "grad_norm": 12.220688819885254,
+      "learning_rate": 7.1081081081081085e-06,
+      "loss": 1.6311,
+      "step": 113
+    },
+    {
+      "epoch": 0.304,
+      "grad_norm": 12.031643867492676,
+      "learning_rate": 7.0810810810810815e-06,
+      "loss": 1.5752,
+      "step": 114
+    },
+    {
+      "epoch": 0.30666666666666664,
+      "grad_norm": 11.963508605957031,
+      "learning_rate": 7.054054054054055e-06,
+      "loss": 1.6123,
+      "step": 115
+    },
+    {
+      "epoch": 0.30933333333333335,
+      "grad_norm": 12.681684494018555,
+      "learning_rate": 7.027027027027028e-06,
+      "loss": 1.6943,
+      "step": 116
+    },
+    {
+      "epoch": 0.312,
+      "grad_norm": 11.903910636901855,
+      "learning_rate": 7e-06,
+      "loss": 1.7237,
+      "step": 117
+    },
+    {
+      "epoch": 0.31466666666666665,
+      "grad_norm": 12.239933967590332,
+      "learning_rate": 6.972972972972973e-06,
+      "loss": 1.532,
+      "step": 118
+    },
+    {
+      "epoch": 0.31733333333333336,
+      "grad_norm": 12.396098136901855,
+      "learning_rate": 6.945945945945947e-06,
+      "loss": 1.6313,
+      "step": 119
+    },
+    {
+      "epoch": 0.32,
+      "grad_norm": 11.794013977050781,
+      "learning_rate": 6.91891891891892e-06,
+      "loss": 1.5266,
+      "step": 120
+    },
+    {
+      "epoch": 0.32,
+      "eval_loss": 1.6251609325408936,
+      "eval_runtime": 1.46,
+      "eval_samples_per_second": 20.548,
+      "eval_steps_per_second": 10.274,
+      "step": 120
+    },
+    {
+      "epoch": 0.32266666666666666,
+      "grad_norm": 10.741290092468262,
+      "learning_rate": 6.891891891891892e-06,
+      "loss": 1.6946,
+      "step": 121
+    },
+    {
+      "epoch": 0.3253333333333333,
+      "grad_norm": 13.47004508972168,
+      "learning_rate": 6.864864864864865e-06,
+      "loss": 1.4668,
+      "step": 122
+    },
+    {
+      "epoch": 0.328,
+      "grad_norm": 12.196053504943848,
+      "learning_rate": 6.837837837837838e-06,
+      "loss": 1.7412,
+      "step": 123
+    },
+    {
+      "epoch": 0.33066666666666666,
+      "grad_norm": 13.029918670654297,
+      "learning_rate": 6.810810810810811e-06,
+      "loss": 1.5352,
+      "step": 124
+    },
+    {
+      "epoch": 0.3333333333333333,
+      "grad_norm": 11.20665454864502,
+      "learning_rate": 6.783783783783784e-06,
+      "loss": 1.5555,
+      "step": 125
+    },
+    {
+      "epoch": 0.336,
+      "grad_norm": 12.214777946472168,
+      "learning_rate": 6.7567567567567575e-06,
+      "loss": 1.5862,
+      "step": 126
+    },
+    {
+      "epoch": 0.33866666666666667,
+      "grad_norm": 13.904444694519043,
+      "learning_rate": 6.729729729729731e-06,
+      "loss": 1.7348,
+      "step": 127
+    },
+    {
+      "epoch": 0.3413333333333333,
+      "grad_norm": 12.65821361541748,
+      "learning_rate": 6.702702702702704e-06,
+      "loss": 1.5627,
+      "step": 128
+    },
+    {
+      "epoch": 0.344,
+      "grad_norm": 12.02226734161377,
+      "learning_rate": 6.675675675675676e-06,
+      "loss": 1.5978,
+      "step": 129
+    },
+    {
+      "epoch": 0.3466666666666667,
+      "grad_norm": 11.53661823272705,
+      "learning_rate": 6.648648648648649e-06,
+      "loss": 1.6756,
+      "step": 130
+    },
+    {
+      "epoch": 0.3466666666666667,
+      "eval_loss": 1.6016688346862793,
+      "eval_runtime": 1.4506,
+      "eval_samples_per_second": 20.681,
+      "eval_steps_per_second": 10.34,
+      "step": 130
+    },
+    {
+      "epoch": 0.34933333333333333,
+      "grad_norm": 13.927865028381348,
+      "learning_rate": 6.621621621621622e-06,
+      "loss": 1.6014,
+      "step": 131
+    },
+    {
+      "epoch": 0.352,
+      "grad_norm": 11.404069900512695,
+      "learning_rate": 6.594594594594595e-06,
+      "loss": 1.6191,
+      "step": 132
+    },
+    {
+      "epoch": 0.3546666666666667,
+      "grad_norm": 12.802854537963867,
+      "learning_rate": 6.567567567567567e-06,
+      "loss": 1.7147,
+      "step": 133
+    },
+    {
+      "epoch": 0.35733333333333334,
+      "grad_norm": 13.293572425842285,
+      "learning_rate": 6.540540540540541e-06,
+      "loss": 1.7999,
+      "step": 134
+    },
+    {
+      "epoch": 0.36,
+      "grad_norm": 14.33857536315918,
+      "learning_rate": 6.513513513513514e-06,
+      "loss": 1.7601,
+      "step": 135
+    },
+    {
+      "epoch": 0.3626666666666667,
+      "grad_norm": 11.54388427734375,
+      "learning_rate": 6.486486486486487e-06,
+      "loss": 1.5501,
+      "step": 136
+    },
+    {
+      "epoch": 0.36533333333333334,
+      "grad_norm": 11.567480087280273,
+      "learning_rate": 6.45945945945946e-06,
+      "loss": 1.6299,
+      "step": 137
+    },
+    {
+      "epoch": 0.368,
+      "grad_norm": 11.44282341003418,
+      "learning_rate": 6.432432432432433e-06,
+      "loss": 1.5726,
+      "step": 138
+    },
+    {
+      "epoch": 0.37066666666666664,
+      "grad_norm": 14.322295188903809,
+      "learning_rate": 6.405405405405406e-06,
+      "loss": 1.6423,
+      "step": 139
+    },
+    {
+      "epoch": 0.37333333333333335,
+      "grad_norm": 11.916459083557129,
+      "learning_rate": 6.378378378378379e-06,
+      "loss": 1.6311,
+      "step": 140
+    },
+    {
+      "epoch": 0.37333333333333335,
+      "eval_loss": 1.5860799551010132,
+      "eval_runtime": 1.4507,
+      "eval_samples_per_second": 20.679,
+      "eval_steps_per_second": 10.34,
+      "step": 140
+    },
+    {
+      "epoch": 0.376,
+      "grad_norm": 11.447466850280762,
+      "learning_rate": 6.351351351351351e-06,
+      "loss": 1.5076,
+      "step": 141
+    },
+    {
+      "epoch": 0.37866666666666665,
+      "grad_norm": 10.863157272338867,
+      "learning_rate": 6.324324324324325e-06,
+      "loss": 1.5188,
+      "step": 142
+    },
+    {
+      "epoch": 0.38133333333333336,
+      "grad_norm": 11.271020889282227,
+      "learning_rate": 6.297297297297298e-06,
+      "loss": 1.6336,
+      "step": 143
+    },
+    {
+      "epoch": 0.384,
+      "grad_norm": 11.583234786987305,
+      "learning_rate": 6.270270270270271e-06,
+      "loss": 1.4692,
+      "step": 144
+    },
+    {
+      "epoch": 0.38666666666666666,
+      "grad_norm": 12.622394561767578,
+      "learning_rate": 6.243243243243243e-06,
+      "loss": 1.3161,
+      "step": 145
+    },
+    {
+      "epoch": 0.3893333333333333,
+      "grad_norm": 10.768049240112305,
+      "learning_rate": 6.2162162162162164e-06,
+      "loss": 1.6118,
+      "step": 146
+    },
+    {
+      "epoch": 0.392,
+      "grad_norm": 10.967697143554688,
+      "learning_rate": 6.1891891891891895e-06,
+      "loss": 1.5251,
+      "step": 147
+    },
+    {
+      "epoch": 0.39466666666666667,
+      "grad_norm": 11.763079643249512,
+      "learning_rate": 6.162162162162163e-06,
+      "loss": 1.6997,
+      "step": 148
+    },
+    {
+      "epoch": 0.3973333333333333,
+      "grad_norm": 10.566551208496094,
+      "learning_rate": 6.135135135135135e-06,
+      "loss": 1.4984,
+      "step": 149
+    },
+    {
+      "epoch": 0.4,
+      "grad_norm": 13.371134757995605,
+      "learning_rate": 6.108108108108109e-06,
+      "loss": 1.7244,
+      "step": 150
+    },
+    {
+      "epoch": 0.4,
+      "eval_loss": 1.5736008882522583,
+      "eval_runtime": 1.4456,
+      "eval_samples_per_second": 20.752,
+      "eval_steps_per_second": 10.376,
+      "step": 150
+    },
+    {
+      "epoch": 0.4026666666666667,
+      "grad_norm": 12.493298530578613,
+      "learning_rate": 6.081081081081082e-06,
+      "loss": 1.5759,
+      "step": 151
+    },
+    {
+      "epoch": 0.4053333333333333,
+      "grad_norm": 12.431512832641602,
+      "learning_rate": 6.054054054054055e-06,
+      "loss": 1.5811,
+      "step": 152
+    },
+    {
+      "epoch": 0.408,
+      "grad_norm": 11.778576850891113,
+      "learning_rate": 6.027027027027027e-06,
+      "loss": 1.5997,
+      "step": 153
+    },
+    {
+      "epoch": 0.4106666666666667,
+      "grad_norm": 12.10274887084961,
+      "learning_rate": 6e-06,
+      "loss": 1.5589,
+      "step": 154
+    },
+    {
+      "epoch": 0.41333333333333333,
+      "grad_norm": 12.245983123779297,
+      "learning_rate": 5.972972972972973e-06,
+      "loss": 1.5864,
+      "step": 155
+    },
+    {
+      "epoch": 0.416,
+      "grad_norm": 12.401384353637695,
+      "learning_rate": 5.945945945945947e-06,
+      "loss": 1.5593,
+      "step": 156
+    },
+    {
+      "epoch": 0.4186666666666667,
+      "grad_norm": 13.845521926879883,
+      "learning_rate": 5.9189189189189185e-06,
+      "loss": 1.4717,
+      "step": 157
+    },
+    {
+      "epoch": 0.42133333333333334,
+      "grad_norm": 11.529600143432617,
+      "learning_rate": 5.8918918918918924e-06,
+      "loss": 1.5599,
+      "step": 158
+    },
+    {
+      "epoch": 0.424,
+      "grad_norm": 11.169657707214355,
+      "learning_rate": 5.8648648648648655e-06,
+      "loss": 1.7053,
+      "step": 159
+    },
+    {
+      "epoch": 0.4266666666666667,
+      "grad_norm": 13.295831680297852,
+      "learning_rate": 5.837837837837839e-06,
+      "loss": 1.4569,
+      "step": 160
+    },
+    {
+      "epoch": 0.4266666666666667,
+      "eval_loss": 1.5643337965011597,
+      "eval_runtime": 1.4526,
+      "eval_samples_per_second": 20.652,
+      "eval_steps_per_second": 10.326,
+      "step": 160
+    },
+    {
+      "epoch": 0.42933333333333334,
+      "grad_norm": 12.037769317626953,
+      "learning_rate": 5.810810810810811e-06,
+      "loss": 1.533,
+      "step": 161
+    },
+    {
+      "epoch": 0.432,
+      "grad_norm": 12.236255645751953,
+      "learning_rate": 5.783783783783784e-06,
+      "loss": 1.5459,
+      "step": 162
+    },
+    {
+      "epoch": 0.43466666666666665,
+      "grad_norm": 12.55140209197998,
+      "learning_rate": 5.756756756756757e-06,
+      "loss": 1.5024,
+      "step": 163
+    },
+    {
+      "epoch": 0.43733333333333335,
+      "grad_norm": 10.870196342468262,
+      "learning_rate": 5.729729729729731e-06,
+      "loss": 1.5159,
+      "step": 164
+    },
+    {
+      "epoch": 0.44,
+      "grad_norm": 11.8145751953125,
+      "learning_rate": 5.702702702702702e-06,
+      "loss": 1.5292,
+      "step": 165
+    },
+    {
+      "epoch": 0.44266666666666665,
+      "grad_norm": 11.710899353027344,
+      "learning_rate": 5.675675675675676e-06,
+      "loss": 1.5724,
+      "step": 166
+    },
+    {
+      "epoch": 0.44533333333333336,
+      "grad_norm": 10.809335708618164,
+      "learning_rate": 5.648648648648649e-06,
+      "loss": 1.5325,
+      "step": 167
+    },
+    {
+      "epoch": 0.448,
+      "grad_norm": 11.874850273132324,
+      "learning_rate": 5.621621621621622e-06,
+      "loss": 1.4954,
+      "step": 168
+    },
+    {
+      "epoch": 0.45066666666666666,
+      "grad_norm": 11.912819862365723,
+      "learning_rate": 5.5945945945945945e-06,
+      "loss": 1.5614,
+      "step": 169
+    },
+    {
+      "epoch": 0.4533333333333333,
+      "grad_norm": 13.068821907043457,
+      "learning_rate": 5.567567567567568e-06,
+      "loss": 1.5157,
+      "step": 170
+    },
+    {
+      "epoch": 0.4533333333333333,
+      "eval_loss": 1.5530024766921997,
+      "eval_runtime": 1.4596,
+      "eval_samples_per_second": 20.554,
+      "eval_steps_per_second": 10.277,
+      "step": 170
+    },
+    {
+      "epoch": 0.456,
+      "grad_norm": 12.24840259552002,
+      "learning_rate": 5.540540540540541e-06,
+      "loss": 1.5693,
+      "step": 171
+    },
+    {
+      "epoch": 0.45866666666666667,
+      "grad_norm": 11.337618827819824,
+      "learning_rate": 5.513513513513515e-06,
+      "loss": 1.5568,
+      "step": 172
+    },
+    {
+      "epoch": 0.4613333333333333,
+      "grad_norm": 10.453991889953613,
+      "learning_rate": 5.486486486486487e-06,
+      "loss": 1.5001,
+      "step": 173
+    },
+    {
+      "epoch": 0.464,
+      "grad_norm": 10.317952156066895,
+      "learning_rate": 5.45945945945946e-06,
+      "loss": 1.4289,
+      "step": 174
+    },
+    {
+      "epoch": 0.4666666666666667,
+      "grad_norm": 11.70666217803955,
+      "learning_rate": 5.432432432432433e-06,
+      "loss": 1.3301,
+      "step": 175
+    },
+    {
+      "epoch": 0.4693333333333333,
+      "grad_norm": 14.784002304077148,
+      "learning_rate": 5.405405405405406e-06,
+      "loss": 1.5322,
+      "step": 176
+    },
+    {
+      "epoch": 0.472,
+      "grad_norm": 10.439934730529785,
+      "learning_rate": 5.378378378378378e-06,
+      "loss": 1.4356,
+      "step": 177
+    },
+    {
+      "epoch": 0.4746666666666667,
+      "grad_norm": 10.800159454345703,
+      "learning_rate": 5.351351351351351e-06,
+      "loss": 1.6718,
+      "step": 178
+    },
+    {
+      "epoch": 0.47733333333333333,
+      "grad_norm": 11.382489204406738,
+      "learning_rate": 5.324324324324324e-06,
+      "loss": 1.6329,
+      "step": 179
+    },
+    {
+      "epoch": 0.48,
+      "grad_norm": 11.744807243347168,
+      "learning_rate": 5.297297297297298e-06,
+      "loss": 1.5418,
+      "step": 180
+    },
+    {
+      "epoch": 0.48,
+      "eval_loss": 1.5468370914459229,
+      "eval_runtime": 1.4494,
+      "eval_samples_per_second": 20.698,
+      "eval_steps_per_second": 10.349,
+      "step": 180
+    },
+    {
+      "epoch": 0.4826666666666667,
+      "grad_norm": 10.518260955810547,
+      "learning_rate": 5.2702702702702705e-06,
+      "loss": 1.579,
+      "step": 181
+    },
+    {
+      "epoch": 0.48533333333333334,
+      "grad_norm": 14.920302391052246,
+      "learning_rate": 5.243243243243244e-06,
+      "loss": 1.5459,
+      "step": 182
+    },
+    {
+      "epoch": 0.488,
+      "grad_norm": 14.005922317504883,
+      "learning_rate": 5.216216216216217e-06,
+      "loss": 1.4835,
+      "step": 183
+    },
+    {
+      "epoch": 0.49066666666666664,
+      "grad_norm": 13.020917892456055,
+      "learning_rate": 5.18918918918919e-06,
+      "loss": 1.5315,
+      "step": 184
+    },
+    {
+      "epoch": 0.49333333333333335,
+      "grad_norm": 11.8460054397583,
+      "learning_rate": 5.162162162162162e-06,
+      "loss": 1.4567,
+      "step": 185
+    },
+    {
+      "epoch": 0.496,
+      "grad_norm": 12.585269927978516,
+      "learning_rate": 5.135135135135135e-06,
+      "loss": 1.5409,
+      "step": 186
+    },
+    {
+      "epoch": 0.49866666666666665,
+      "grad_norm": 10.893098831176758,
+      "learning_rate": 5.108108108108108e-06,
+      "loss": 1.5523,
+      "step": 187
+    },
+    {
+      "epoch": 0.5013333333333333,
+      "grad_norm": 13.312434196472168,
+      "learning_rate": 5.081081081081082e-06,
+      "loss": 1.408,
+      "step": 188
+    },
+    {
+      "epoch": 0.504,
+      "grad_norm": 11.251080513000488,
+      "learning_rate": 5.054054054054054e-06,
+      "loss": 1.559,
+      "step": 189
+    },
+    {
+      "epoch": 0.5066666666666667,
+      "grad_norm": 11.128360748291016,
+      "learning_rate": 5.027027027027027e-06,
+      "loss": 1.6048,
+      "step": 190
+    },
+    {
+      "epoch": 0.5066666666666667,
+      "eval_loss": 1.5399889945983887,
+      "eval_runtime": 1.4761,
+      "eval_samples_per_second": 20.324,
+      "eval_steps_per_second": 10.162,
+      "step": 190
+    },
+    {
+      "epoch": 0.5093333333333333,
+      "grad_norm": 11.423062324523926,
+      "learning_rate": 5e-06,
+      "loss": 1.6389,
+      "step": 191
+    },
+    {
+      "epoch": 0.512,
+      "grad_norm": 11.651025772094727,
+      "learning_rate": 4.9729729729729735e-06,
+      "loss": 1.5658,
+      "step": 192
+    },
+    {
+      "epoch": 0.5146666666666667,
+      "grad_norm": 11.046283721923828,
+      "learning_rate": 4.9459459459459466e-06,
+      "loss": 1.4467,
+      "step": 193
+    },
+    {
+      "epoch": 0.5173333333333333,
+      "grad_norm": 12.875414848327637,
+      "learning_rate": 4.91891891891892e-06,
+      "loss": 1.5826,
+      "step": 194
+    },
+    {
+      "epoch": 0.52,
+      "grad_norm": 12.292638778686523,
+      "learning_rate": 4.891891891891893e-06,
+      "loss": 1.483,
+      "step": 195
+    },
+    {
+      "epoch": 0.5226666666666666,
+      "grad_norm": 11.317679405212402,
+      "learning_rate": 4.864864864864866e-06,
+      "loss": 1.6073,
+      "step": 196
+    },
+    {
+      "epoch": 0.5253333333333333,
+      "grad_norm": 12.65168571472168,
+      "learning_rate": 4.837837837837838e-06,
+      "loss": 1.5852,
+      "step": 197
+    },
+    {
+      "epoch": 0.528,
+      "grad_norm": 14.210925102233887,
+      "learning_rate": 4.810810810810811e-06,
+      "loss": 1.6664,
+      "step": 198
+    },
+    {
+      "epoch": 0.5306666666666666,
+      "grad_norm": 12.766249656677246,
+      "learning_rate": 4.783783783783784e-06,
+      "loss": 1.6201,
+      "step": 199
+    },
+    {
+      "epoch": 0.5333333333333333,
+      "grad_norm": 12.087968826293945,
+      "learning_rate": 4.756756756756757e-06,
+      "loss": 1.5182,
+      "step": 200
+    },
+    {
+      "epoch": 0.5333333333333333,
+      "eval_loss": 1.5342140197753906,
+      "eval_runtime": 1.4482,
+      "eval_samples_per_second": 20.716,
+      "eval_steps_per_second": 10.358,
+      "step": 200
+    },
+    {
+      "epoch": 0.536,
+      "grad_norm": 14.878036499023438,
+      "learning_rate": 4.72972972972973e-06,
+      "loss": 1.6464,
+      "step": 201
+    },
+    {
+      "epoch": 0.5386666666666666,
+      "grad_norm": 11.994093894958496,
+      "learning_rate": 4.702702702702703e-06,
+      "loss": 1.4874,
+      "step": 202
+    },
+    {
+      "epoch": 0.5413333333333333,
+      "grad_norm": 16.825557708740234,
+      "learning_rate": 4.675675675675676e-06,
+      "loss": 1.6089,
+      "step": 203
+    },
+    {
+      "epoch": 0.544,
+      "grad_norm": 10.64698314666748,
+      "learning_rate": 4.6486486486486495e-06,
+      "loss": 1.5708,
+      "step": 204
+    },
+    {
+      "epoch": 0.5466666666666666,
+      "grad_norm": 12.136466979980469,
+      "learning_rate": 4.621621621621622e-06,
+      "loss": 1.686,
+      "step": 205
+    },
+    {
+      "epoch": 0.5493333333333333,
+      "grad_norm": 11.439790725708008,
+      "learning_rate": 4.594594594594596e-06,
+      "loss": 1.6251,
+      "step": 206
+    },
+    {
+      "epoch": 0.552,
+      "grad_norm": 11.850600242614746,
+      "learning_rate": 4.567567567567568e-06,
+      "loss": 1.5279,
+      "step": 207
+    },
+    {
+      "epoch": 0.5546666666666666,
+      "grad_norm": 11.281230926513672,
+      "learning_rate": 4.540540540540541e-06,
+      "loss": 1.4779,
+      "step": 208
+    },
+    {
+      "epoch": 0.5573333333333333,
+      "grad_norm": 11.568439483642578,
+      "learning_rate": 4.513513513513514e-06,
+      "loss": 1.717,
+      "step": 209
+    },
+    {
+      "epoch": 0.56,
+      "grad_norm": 10.368412017822266,
+      "learning_rate": 4.486486486486487e-06,
+      "loss": 1.5628,
+      "step": 210
+    },
+    {
+      "epoch": 0.56,
+      "eval_loss": 1.529217004776001,
+      "eval_runtime": 1.4476,
+      "eval_samples_per_second": 20.724,
+      "eval_steps_per_second": 10.362,
+      "step": 210
+    },
+    {
+      "epoch": 0.5626666666666666,
+      "grad_norm": 12.218080520629883,
+      "learning_rate": 4.45945945945946e-06,
+      "loss": 1.5924,
+      "step": 211
+    },
+    {
+      "epoch": 0.5653333333333334,
+      "grad_norm": 10.668429374694824,
+      "learning_rate": 4.432432432432433e-06,
+      "loss": 1.5156,
+      "step": 212
+    },
+    {
+      "epoch": 0.568,
+      "grad_norm": 10.990684509277344,
+      "learning_rate": 4.4054054054054054e-06,
+      "loss": 1.6046,
+      "step": 213
+    },
+    {
+      "epoch": 0.5706666666666667,
+      "grad_norm": 12.811287879943848,
+      "learning_rate": 4.378378378378379e-06,
+      "loss": 1.5945,
+      "step": 214
+    },
+    {
+      "epoch": 0.5733333333333334,
+      "grad_norm": 11.771529197692871,
+      "learning_rate": 4.351351351351352e-06,
+      "loss": 1.5792,
+      "step": 215
+    },
+    {
+      "epoch": 0.576,
+      "grad_norm": 11.0822114944458,
+      "learning_rate": 4.324324324324325e-06,
+      "loss": 1.6252,
+      "step": 216
+    },
+    {
+      "epoch": 0.5786666666666667,
+      "grad_norm": 11.860054016113281,
+      "learning_rate": 4.297297297297298e-06,
+      "loss": 1.4526,
+      "step": 217
+    },
+    {
+      "epoch": 0.5813333333333334,
+      "grad_norm": 11.882978439331055,
+      "learning_rate": 4.270270270270271e-06,
+      "loss": 1.3806,
+      "step": 218
+    },
+    {
+      "epoch": 0.584,
+      "grad_norm": 11.159905433654785,
+      "learning_rate": 4.243243243243244e-06,
+      "loss": 1.5448,
+      "step": 219
+    },
+    {
+      "epoch": 0.5866666666666667,
+      "grad_norm": 12.598294258117676,
+      "learning_rate": 4.216216216216217e-06,
+      "loss": 1.531,
+      "step": 220
+    },
+    {
+      "epoch": 0.5866666666666667,
+      "eval_loss": 1.5237351655960083,
+      "eval_runtime": 1.505,
+      "eval_samples_per_second": 19.934,
+      "eval_steps_per_second": 9.967,
+      "step": 220
+    },
+    {
+      "epoch": 0.5893333333333334,
+      "grad_norm": 12.2394437789917,
+      "learning_rate": 4.189189189189189e-06,
+      "loss": 1.4149,
+      "step": 221
+    },
+    {
+      "epoch": 0.592,
+      "grad_norm": 11.620826721191406,
+      "learning_rate": 4.162162162162163e-06,
+      "loss": 1.6498,
+      "step": 222
+    },
+    {
+      "epoch": 0.5946666666666667,
+      "grad_norm": 13.30986213684082,
+      "learning_rate": 4.135135135135135e-06,
+      "loss": 1.6125,
+      "step": 223
+    },
+    {
+      "epoch": 0.5973333333333334,
+      "grad_norm": 10.7543363571167,
+      "learning_rate": 4.108108108108108e-06,
+      "loss": 1.5335,
+      "step": 224
+    },
+    {
+      "epoch": 0.6,
+      "grad_norm": 12.820805549621582,
+      "learning_rate": 4.0810810810810815e-06,
+      "loss": 1.4477,
+      "step": 225
+    },
+    {
+      "epoch": 0.6026666666666667,
+      "grad_norm": 10.250899314880371,
+      "learning_rate": 4.0540540540540545e-06,
+      "loss": 1.5308,
+      "step": 226
+    },
+    {
+      "epoch": 0.6053333333333333,
+      "grad_norm": 11.902603149414062,
+      "learning_rate": 4.027027027027028e-06,
+      "loss": 1.5168,
+      "step": 227
+    },
+    {
+      "epoch": 0.608,
+      "grad_norm": 11.358914375305176,
+      "learning_rate": 4.000000000000001e-06,
+      "loss": 1.5414,
+      "step": 228
+    },
+    {
+      "epoch": 0.6106666666666667,
+      "grad_norm": 10.709514617919922,
+      "learning_rate": 3.972972972972973e-06,
+      "loss": 1.4812,
+      "step": 229
+    },
+    {
+      "epoch": 0.6133333333333333,
+      "grad_norm": 10.724623680114746,
+      "learning_rate": 3.945945945945947e-06,
+      "loss": 1.5425,
+      "step": 230
+    },
+    {
+      "epoch": 0.6133333333333333,
+      "eval_loss": 1.5228849649429321,
+      "eval_runtime": 1.4436,
+      "eval_samples_per_second": 20.781,
+      "eval_steps_per_second": 10.39,
+      "step": 230
+    },
+    {
+      "epoch": 0.616,
+      "grad_norm": 11.70534896850586,
+      "learning_rate": 3.918918918918919e-06,
+      "loss": 1.5146,
+      "step": 231
+    },
+    {
+      "epoch": 0.6186666666666667,
+      "grad_norm": 9.500643730163574,
+      "learning_rate": 3.891891891891892e-06,
+      "loss": 1.4185,
+      "step": 232
+    },
+    {
+      "epoch": 0.6213333333333333,
+      "grad_norm": 11.755574226379395,
+      "learning_rate": 3.864864864864865e-06,
+      "loss": 1.5805,
+      "step": 233
+    },
+    {
+      "epoch": 0.624,
+      "grad_norm": 11.309388160705566,
+      "learning_rate": 3.837837837837838e-06,
+      "loss": 1.5481,
+      "step": 234
+    },
+    {
+      "epoch": 0.6266666666666667,
+      "grad_norm": 10.823451042175293,
+      "learning_rate": 3.810810810810811e-06,
+      "loss": 1.4996,
+      "step": 235
+    },
+    {
+      "epoch": 0.6293333333333333,
+      "grad_norm": 10.33895492553711,
+      "learning_rate": 3.7837837837837844e-06,
+      "loss": 1.5545,
+      "step": 236
+    },
+    {
+      "epoch": 0.632,
+      "grad_norm": 12.343066215515137,
+      "learning_rate": 3.756756756756757e-06,
+      "loss": 1.4629,
+      "step": 237
+    },
+    {
+      "epoch": 0.6346666666666667,
+      "grad_norm": 10.669163703918457,
+      "learning_rate": 3.72972972972973e-06,
+      "loss": 1.4943,
+      "step": 238
+    },
+    {
+      "epoch": 0.6373333333333333,
+      "grad_norm": 19.491622924804688,
+      "learning_rate": 3.7027027027027028e-06,
+      "loss": 1.4613,
+      "step": 239
+    },
+    {
+      "epoch": 0.64,
+      "grad_norm": 11.280375480651855,
+      "learning_rate": 3.6756756756756763e-06,
+      "loss": 1.5251,
+      "step": 240
+    },
+    {
+      "epoch": 0.64,
+      "eval_loss": 1.5189076662063599,
+      "eval_runtime": 1.4647,
+      "eval_samples_per_second": 20.482,
+      "eval_steps_per_second": 10.241,
+      "step": 240
+    },
+    {
+      "epoch": 0.6426666666666667,
+      "grad_norm": 11.986125946044922,
+      "learning_rate": 3.648648648648649e-06,
+      "loss": 1.5283,
+      "step": 241
+    },
+    {
+      "epoch": 0.6453333333333333,
+      "grad_norm": 11.676025390625,
+      "learning_rate": 3.621621621621622e-06,
+      "loss": 1.5087,
+      "step": 242
+    },
+    {
+      "epoch": 0.648,
+      "grad_norm": 11.277451515197754,
+      "learning_rate": 3.5945945945945946e-06,
+      "loss": 1.5804,
+      "step": 243
+    },
+    {
+      "epoch": 0.6506666666666666,
+      "grad_norm": 9.987862586975098,
+      "learning_rate": 3.567567567567568e-06,
+      "loss": 1.5514,
+      "step": 244
+    },
+    {
+      "epoch": 0.6533333333333333,
+      "grad_norm": 11.72730827331543,
+      "learning_rate": 3.5405405405405408e-06,
+      "loss": 1.51,
+      "step": 245
+    },
+    {
+      "epoch": 0.656,
+      "grad_norm": 11.331317901611328,
+      "learning_rate": 3.513513513513514e-06,
+      "loss": 1.4503,
+      "step": 246
+    },
+    {
+      "epoch": 0.6586666666666666,
+      "grad_norm": 10.42099666595459,
+      "learning_rate": 3.4864864864864865e-06,
+      "loss": 1.4682,
+      "step": 247
+    },
+    {
+      "epoch": 0.6613333333333333,
+      "grad_norm": 12.430152893066406,
+      "learning_rate": 3.45945945945946e-06,
+      "loss": 1.5126,
+      "step": 248
+    },
+    {
+      "epoch": 0.664,
+      "grad_norm": 12.36617660522461,
+      "learning_rate": 3.4324324324324326e-06,
+      "loss": 1.5373,
+      "step": 249
+    },
+    {
+      "epoch": 0.6666666666666666,
+      "grad_norm": 11.66644287109375,
+      "learning_rate": 3.4054054054054057e-06,
+      "loss": 1.5439,
+      "step": 250
+    },
+    {
+      "epoch": 0.6666666666666666,
+      "eval_loss": 1.5144615173339844,
+      "eval_runtime": 1.4468,
+      "eval_samples_per_second": 20.735,
+      "eval_steps_per_second": 10.368,
+      "step": 250
+    },
+    {
+      "epoch": 0.6693333333333333,
+      "grad_norm": 11.48746395111084,
+      "learning_rate": 3.3783783783783788e-06,
+      "loss": 1.4353,
+      "step": 251
+    },
+    {
+      "epoch": 0.672,
+      "grad_norm": 11.128070831298828,
+      "learning_rate": 3.351351351351352e-06,
+      "loss": 1.5068,
+      "step": 252
+    },
+    {
+      "epoch": 0.6746666666666666,
+      "grad_norm": 12.601024627685547,
+      "learning_rate": 3.3243243243243245e-06,
+      "loss": 1.5039,
+      "step": 253
+    },
+    {
+      "epoch": 0.6773333333333333,
+      "grad_norm": 11.611284255981445,
+      "learning_rate": 3.2972972972972976e-06,
+      "loss": 1.5565,
+      "step": 254
+    },
+    {
+      "epoch": 0.68,
+      "grad_norm": 12.905797958374023,
+      "learning_rate": 3.2702702702702706e-06,
+      "loss": 1.5206,
+      "step": 255
+    },
+    {
+      "epoch": 0.6826666666666666,
+      "grad_norm": 12.170700073242188,
+      "learning_rate": 3.2432432432432437e-06,
+      "loss": 1.2695,
+      "step": 256
+    },
+    {
+      "epoch": 0.6853333333333333,
+      "grad_norm": 12.405909538269043,
+      "learning_rate": 3.2162162162162164e-06,
+      "loss": 1.5179,
+      "step": 257
+    },
+    {
+      "epoch": 0.688,
+      "grad_norm": 13.01850414276123,
+      "learning_rate": 3.1891891891891894e-06,
+      "loss": 1.4435,
+      "step": 258
+    },
+    {
+      "epoch": 0.6906666666666667,
+      "grad_norm": 11.52682113647461,
+      "learning_rate": 3.1621621621621625e-06,
+      "loss": 1.3751,
+      "step": 259
+    },
+    {
+      "epoch": 0.6933333333333334,
+      "grad_norm": 10.300020217895508,
+      "learning_rate": 3.1351351351351356e-06,
+      "loss": 1.4654,
+      "step": 260
+    },
+    {
+      "epoch": 0.6933333333333334,
+      "eval_loss": 1.5129177570343018,
+      "eval_runtime": 1.4415,
+      "eval_samples_per_second": 20.811,
+      "eval_steps_per_second": 10.406,
+      "step": 260
+    },
+    {
+      "epoch": 0.696,
+      "grad_norm": 11.269693374633789,
+      "learning_rate": 3.1081081081081082e-06,
+      "loss": 1.6435,
+      "step": 261
+    },
+    {
+      "epoch": 0.6986666666666667,
+      "grad_norm": 11.508820533752441,
+      "learning_rate": 3.0810810810810817e-06,
+      "loss": 1.4721,
+      "step": 262
+    },
+    {
+      "epoch": 0.7013333333333334,
+      "grad_norm": 10.975653648376465,
+      "learning_rate": 3.0540540540540544e-06,
+      "loss": 1.672,
+      "step": 263
+    },
+    {
+      "epoch": 0.704,
+      "grad_norm": 13.16110897064209,
+      "learning_rate": 3.0270270270270274e-06,
+      "loss": 1.4838,
+      "step": 264
+    },
+    {
+      "epoch": 0.7066666666666667,
+      "grad_norm": 13.65386962890625,
+      "learning_rate": 3e-06,
+      "loss": 1.618,
+      "step": 265
+    },
+    {
+      "epoch": 0.7093333333333334,
+      "grad_norm": 12.109015464782715,
+      "learning_rate": 2.9729729729729736e-06,
+      "loss": 1.4518,
+      "step": 266
+    },
+    {
+      "epoch": 0.712,
+      "grad_norm": 11.558544158935547,
+      "learning_rate": 2.9459459459459462e-06,
+      "loss": 1.4772,
+      "step": 267
+    },
+    {
+      "epoch": 0.7146666666666667,
+      "grad_norm": 12.302791595458984,
+      "learning_rate": 2.9189189189189193e-06,
+      "loss": 1.6029,
+      "step": 268
+    },
+    {
+      "epoch": 0.7173333333333334,
+      "grad_norm": 12.268084526062012,
+      "learning_rate": 2.891891891891892e-06,
+      "loss": 1.4477,
+      "step": 269
+    },
+    {
+      "epoch": 0.72,
+      "grad_norm": 11.1484375,
+      "learning_rate": 2.8648648648648654e-06,
+      "loss": 1.6031,
+      "step": 270
+    },
+    {
+      "epoch": 0.72,
+      "eval_loss": 1.5090078115463257,
+      "eval_runtime": 1.4443,
+      "eval_samples_per_second": 20.772,
+      "eval_steps_per_second": 10.386,
+      "step": 270
+    },
+    {
+      "epoch": 0.7226666666666667,
+      "grad_norm": 10.918089866638184,
+      "learning_rate": 2.837837837837838e-06,
+      "loss": 1.3471,
+      "step": 271
+    },
+    {
+      "epoch": 0.7253333333333334,
+      "grad_norm": 12.327642440795898,
+      "learning_rate": 2.810810810810811e-06,
+      "loss": 1.562,
+      "step": 272
+    },
+    {
+      "epoch": 0.728,
+      "grad_norm": 11.135960578918457,
+      "learning_rate": 2.783783783783784e-06,
+      "loss": 1.5001,
+      "step": 273
+    },
+    {
+      "epoch": 0.7306666666666667,
+      "grad_norm": 11.357678413391113,
+      "learning_rate": 2.7567567567567573e-06,
+      "loss": 1.4937,
+      "step": 274
+    },
+    {
+      "epoch": 0.7333333333333333,
+      "grad_norm": 11.794831275939941,
+      "learning_rate": 2.72972972972973e-06,
+      "loss": 1.4711,
+      "step": 275
+    },
+    {
+      "epoch": 0.736,
+      "grad_norm": 12.003959655761719,
+      "learning_rate": 2.702702702702703e-06,
+      "loss": 1.4683,
+      "step": 276
+    },
+    {
+      "epoch": 0.7386666666666667,
+      "grad_norm": 11.9353666305542,
+      "learning_rate": 2.6756756756756757e-06,
+      "loss": 1.4963,
+      "step": 277
+    },
+    {
+      "epoch": 0.7413333333333333,
+      "grad_norm": 10.868412971496582,
+      "learning_rate": 2.648648648648649e-06,
+      "loss": 1.4788,
+      "step": 278
+    },
+    {
+      "epoch": 0.744,
+      "grad_norm": 10.403037071228027,
+      "learning_rate": 2.621621621621622e-06,
+      "loss": 1.4361,
+      "step": 279
+    },
+    {
+      "epoch": 0.7466666666666667,
+      "grad_norm": 14.62988567352295,
+      "learning_rate": 2.594594594594595e-06,
+      "loss": 1.406,
+      "step": 280
+    },
+    {
+      "epoch": 0.7466666666666667,
+      "eval_loss": 1.5082733631134033,
+      "eval_runtime": 1.4412,
+      "eval_samples_per_second": 20.816,
+      "eval_steps_per_second": 10.408,
+      "step": 280
+    },
+    {
+      "epoch": 0.7493333333333333,
+      "grad_norm": 11.512578964233398,
+      "learning_rate": 2.5675675675675675e-06,
+      "loss": 1.5055,
+      "step": 281
+    },
+    {
+      "epoch": 0.752,
+      "grad_norm": 12.163981437683105,
+      "learning_rate": 2.540540540540541e-06,
+      "loss": 1.5691,
+      "step": 282
+    },
+    {
+      "epoch": 0.7546666666666667,
+      "grad_norm": 12.146513938903809,
+      "learning_rate": 2.5135135135135137e-06,
+      "loss": 1.3984,
+      "step": 283
+    },
+    {
+      "epoch": 0.7573333333333333,
+      "grad_norm": 12.473353385925293,
+      "learning_rate": 2.4864864864864867e-06,
+      "loss": 1.5503,
+      "step": 284
+    },
+    {
+      "epoch": 0.76,
+      "grad_norm": 11.867709159851074,
+      "learning_rate": 2.45945945945946e-06,
+      "loss": 1.4414,
+      "step": 285
+    },
+    {
+      "epoch": 0.7626666666666667,
+      "grad_norm": 10.952685356140137,
+      "learning_rate": 2.432432432432433e-06,
+      "loss": 1.5349,
+      "step": 286
+    },
+    {
+      "epoch": 0.7653333333333333,
+      "grad_norm": 11.330965042114258,
+      "learning_rate": 2.4054054054054055e-06,
+      "loss": 1.5057,
+      "step": 287
+    },
+    {
+      "epoch": 0.768,
+      "grad_norm": 12.43771743774414,
+      "learning_rate": 2.3783783783783786e-06,
+      "loss": 1.5275,
+      "step": 288
+    },
+    {
+      "epoch": 0.7706666666666667,
+      "grad_norm": 10.562946319580078,
+      "learning_rate": 2.3513513513513517e-06,
+      "loss": 1.3893,
+      "step": 289
+    },
+    {
+      "epoch": 0.7733333333333333,
+      "grad_norm": 12.457465171813965,
+      "learning_rate": 2.3243243243243247e-06,
+      "loss": 1.4675,
+      "step": 290
+    },
+    {
+      "epoch": 0.7733333333333333,
+      "eval_loss": 1.505977988243103,
+      "eval_runtime": 1.4423,
+      "eval_samples_per_second": 20.801,
+      "eval_steps_per_second": 10.4,
+      "step": 290
+    },
+    {
+      "epoch": 0.776,
+      "grad_norm": 11.270442008972168,
+      "learning_rate": 2.297297297297298e-06,
+      "loss": 1.5013,
+      "step": 291
+    },
+    {
+      "epoch": 0.7786666666666666,
+      "grad_norm": 13.198526382446289,
+      "learning_rate": 2.2702702702702705e-06,
+      "loss": 1.4414,
+      "step": 292
+    },
+    {
+      "epoch": 0.7813333333333333,
+      "grad_norm": 11.611175537109375,
+      "learning_rate": 2.2432432432432435e-06,
+      "loss": 1.4635,
+      "step": 293
+    },
+    {
+      "epoch": 0.784,
+      "grad_norm": 11.79445743560791,
+      "learning_rate": 2.2162162162162166e-06,
+      "loss": 1.5512,
+      "step": 294
+    },
+    {
+      "epoch": 0.7866666666666666,
+      "grad_norm": 10.822488784790039,
+      "learning_rate": 2.1891891891891897e-06,
+      "loss": 1.5688,
+      "step": 295
+    },
+    {
+      "epoch": 0.7893333333333333,
+      "grad_norm": 10.623988151550293,
+      "learning_rate": 2.1621621621621623e-06,
+      "loss": 1.3992,
+      "step": 296
+    },
+    {
+      "epoch": 0.792,
+      "grad_norm": 10.979094505310059,
+      "learning_rate": 2.1351351351351354e-06,
+      "loss": 1.7058,
+      "step": 297
+    },
+    {
+      "epoch": 0.7946666666666666,
+      "grad_norm": 11.987638473510742,
+      "learning_rate": 2.1081081081081085e-06,
+      "loss": 1.4855,
+      "step": 298
+    },
+    {
+      "epoch": 0.7973333333333333,
+      "grad_norm": 13.160965919494629,
+      "learning_rate": 2.0810810810810815e-06,
+      "loss": 1.7222,
+      "step": 299
+    },
+    {
+      "epoch": 0.8,
+      "grad_norm": 12.35315990447998,
+      "learning_rate": 2.054054054054054e-06,
+      "loss": 1.4974,
+      "step": 300
+    },
+    {
+      "epoch": 0.8,
+      "eval_loss": 1.5055415630340576,
+      "eval_runtime": 1.4432,
+      "eval_samples_per_second": 20.787,
+      "eval_steps_per_second": 10.393,
+      "step": 300
+    },
+    {
+      "epoch": 0.8026666666666666,
+      "grad_norm": 12.041595458984375,
+      "learning_rate": 2.0270270270270273e-06,
+      "loss": 1.5184,
+      "step": 301
+    },
+    {
+      "epoch": 0.8053333333333333,
+      "grad_norm": 11.05543327331543,
+      "learning_rate": 2.0000000000000003e-06,
+      "loss": 1.4602,
+      "step": 302
+    },
+    {
+      "epoch": 0.808,
+      "grad_norm": 11.678079605102539,
+      "learning_rate": 1.9729729729729734e-06,
+      "loss": 1.5511,
+      "step": 303
+    },
+    {
+      "epoch": 0.8106666666666666,
+      "grad_norm": 12.200875282287598,
+      "learning_rate": 1.945945945945946e-06,
+      "loss": 1.4688,
+      "step": 304
+    },
+    {
+      "epoch": 0.8133333333333334,
+      "grad_norm": 10.824845314025879,
+      "learning_rate": 1.918918918918919e-06,
+      "loss": 1.5101,
+      "step": 305
+    },
+    {
+      "epoch": 0.816,
+      "grad_norm": 10.230807304382324,
+      "learning_rate": 1.8918918918918922e-06,
+      "loss": 1.425,
+      "step": 306
+    },
+    {
+      "epoch": 0.8186666666666667,
+      "grad_norm": 11.947959899902344,
+      "learning_rate": 1.864864864864865e-06,
+      "loss": 1.57,
+      "step": 307
+    },
+    {
+      "epoch": 0.8213333333333334,
+      "grad_norm": 13.433063507080078,
+      "learning_rate": 1.8378378378378381e-06,
+      "loss": 1.5816,
+      "step": 308
+    },
+    {
+      "epoch": 0.824,
+      "grad_norm": 10.70764446258545,
+      "learning_rate": 1.810810810810811e-06,
+      "loss": 1.519,
+      "step": 309
+    },
+    {
+      "epoch": 0.8266666666666667,
+      "grad_norm": 11.407464981079102,
+      "learning_rate": 1.783783783783784e-06,
+      "loss": 1.4903,
+      "step": 310
+    },
+    {
+      "epoch": 0.8266666666666667,
+      "eval_loss": 1.5049265623092651,
+      "eval_runtime": 1.4452,
+      "eval_samples_per_second": 20.758,
+      "eval_steps_per_second": 10.379,
+      "step": 310
+    },
+    {
+      "epoch": 0.8293333333333334,
+      "grad_norm": 11.038989067077637,
+      "learning_rate": 1.756756756756757e-06,
+      "loss": 1.5449,
+      "step": 311
+    },
+    {
+      "epoch": 0.832,
+      "grad_norm": 10.891145706176758,
+      "learning_rate": 1.72972972972973e-06,
+      "loss": 1.4414,
+      "step": 312
+    },
+    {
+      "epoch": 0.8346666666666667,
+      "grad_norm": 11.190020561218262,
+      "learning_rate": 1.7027027027027028e-06,
+      "loss": 1.5704,
+      "step": 313
+    },
+    {
+      "epoch": 0.8373333333333334,
+      "grad_norm": 11.971606254577637,
+      "learning_rate": 1.675675675675676e-06,
+      "loss": 1.5643,
+      "step": 314
+    },
+    {
+      "epoch": 0.84,
+      "grad_norm": 10.959304809570312,
+      "learning_rate": 1.6486486486486488e-06,
+      "loss": 1.3744,
+      "step": 315
+    },
+    {
+      "epoch": 0.8426666666666667,
+      "grad_norm": 11.642538070678711,
+      "learning_rate": 1.6216216216216219e-06,
+      "loss": 1.581,
+      "step": 316
+    },
+    {
+      "epoch": 0.8453333333333334,
+      "grad_norm": 11.766486167907715,
+      "learning_rate": 1.5945945945945947e-06,
+      "loss": 1.4898,
+      "step": 317
+    },
+    {
+      "epoch": 0.848,
+      "grad_norm": 12.002728462219238,
+      "learning_rate": 1.5675675675675678e-06,
+      "loss": 1.5529,
+      "step": 318
+    },
+    {
+      "epoch": 0.8506666666666667,
+      "grad_norm": 11.168254852294922,
+      "learning_rate": 1.5405405405405409e-06,
+      "loss": 1.5576,
+      "step": 319
+    },
+    {
+      "epoch": 0.8533333333333334,
+      "grad_norm": 12.34257698059082,
+      "learning_rate": 1.5135135135135137e-06,
+      "loss": 1.379,
+      "step": 320
+    },
+    {
+      "epoch": 0.8533333333333334,
+      "eval_loss": 1.5034152269363403,
+      "eval_runtime": 1.4462,
+      "eval_samples_per_second": 20.744,
+      "eval_steps_per_second": 10.372,
+      "step": 320
+    },
+    {
+      "epoch": 0.856,
+      "grad_norm": 12.051098823547363,
+      "learning_rate": 1.4864864864864868e-06,
+      "loss": 1.5729,
+      "step": 321
+    },
+    {
+      "epoch": 0.8586666666666667,
+      "grad_norm": 10.446905136108398,
+      "learning_rate": 1.4594594594594596e-06,
+      "loss": 1.4935,
+      "step": 322
+    },
+    {
+      "epoch": 0.8613333333333333,
+      "grad_norm": 12.018199920654297,
+      "learning_rate": 1.4324324324324327e-06,
+      "loss": 1.5905,
+      "step": 323
+    },
+    {
+      "epoch": 0.864,
+      "grad_norm": 12.235658645629883,
+      "learning_rate": 1.4054054054054056e-06,
+      "loss": 1.5417,
+      "step": 324
+    },
+    {
+      "epoch": 0.8666666666666667,
+      "grad_norm": 11.277713775634766,
+      "learning_rate": 1.3783783783783786e-06,
+      "loss": 1.5076,
+      "step": 325
+    },
+    {
+      "epoch": 0.8693333333333333,
+      "grad_norm": 10.657114028930664,
+      "learning_rate": 1.3513513513513515e-06,
+      "loss": 1.4326,
+      "step": 326
+    },
+    {
+      "epoch": 0.872,
+      "grad_norm": 12.818350791931152,
+      "learning_rate": 1.3243243243243246e-06,
+      "loss": 1.4677,
+      "step": 327
+    },
+    {
+      "epoch": 0.8746666666666667,
+      "grad_norm": 13.504673957824707,
+      "learning_rate": 1.2972972972972974e-06,
+      "loss": 1.5695,
+      "step": 328
+    },
+    {
+      "epoch": 0.8773333333333333,
+      "grad_norm": 11.200360298156738,
+      "learning_rate": 1.2702702702702705e-06,
+      "loss": 1.4903,
+      "step": 329
+    },
+    {
+      "epoch": 0.88,
+      "grad_norm": 11.851312637329102,
+      "learning_rate": 1.2432432432432434e-06,
+      "loss": 1.4303,
+      "step": 330
+    },
+    {
+      "epoch": 0.88,
+      "eval_loss": 1.5030354261398315,
+      "eval_runtime": 1.6266,
+      "eval_samples_per_second": 18.444,
+      "eval_steps_per_second": 9.222,
+      "step": 330
+    },
+    {
+      "epoch": 0.8826666666666667,
+      "grad_norm": 11.918156623840332,
+      "learning_rate": 1.2162162162162164e-06,
+      "loss": 1.6386,
+      "step": 331
+    },
+    {
+      "epoch": 0.8853333333333333,
+      "grad_norm": 11.092312812805176,
+      "learning_rate": 1.1891891891891893e-06,
+      "loss": 1.5743,
+      "step": 332
+    },
+    {
+      "epoch": 0.888,
+      "grad_norm": 12.29781436920166,
+      "learning_rate": 1.1621621621621624e-06,
+      "loss": 1.4493,
+      "step": 333
+    },
+    {
+      "epoch": 0.8906666666666667,
+      "grad_norm": 12.36745834350586,
+      "learning_rate": 1.1351351351351352e-06,
+      "loss": 1.5803,
+      "step": 334
+    },
+    {
+      "epoch": 0.8933333333333333,
+      "grad_norm": 11.729464530944824,
+      "learning_rate": 1.1081081081081083e-06,
+      "loss": 1.5142,
+      "step": 335
+    },
+    {
+      "epoch": 0.896,
+      "grad_norm": 12.086241722106934,
+      "learning_rate": 1.0810810810810812e-06,
+      "loss": 1.5225,
+      "step": 336
+    },
+    {
+      "epoch": 0.8986666666666666,
+      "grad_norm": 12.04587173461914,
+      "learning_rate": 1.0540540540540542e-06,
+      "loss": 1.5774,
+      "step": 337
+    },
+    {
+      "epoch": 0.9013333333333333,
+      "grad_norm": 11.315978050231934,
+      "learning_rate": 1.027027027027027e-06,
+      "loss": 1.5096,
+      "step": 338
+    },
+    {
+      "epoch": 0.904,
+      "grad_norm": 10.968897819519043,
+      "learning_rate": 1.0000000000000002e-06,
+      "loss": 1.5616,
+      "step": 339
+    },
+    {
+      "epoch": 0.9066666666666666,
+      "grad_norm": 11.676291465759277,
+      "learning_rate": 9.72972972972973e-07,
+      "loss": 1.5634,
+      "step": 340
+    },
+    {
+      "epoch": 0.9066666666666666,
+      "eval_loss": 1.5035680532455444,
+      "eval_runtime": 1.4443,
+      "eval_samples_per_second": 20.771,
+      "eval_steps_per_second": 10.386,
+      "step": 340
+    },
+    {
+      "epoch": 0.9093333333333333,
+      "grad_norm": 11.968234062194824,
+      "learning_rate": 9.459459459459461e-07,
+      "loss": 1.4875,
+      "step": 341
+    },
+    {
+      "epoch": 0.912,
+      "grad_norm": 12.339000701904297,
+      "learning_rate": 9.189189189189191e-07,
+      "loss": 1.661,
+      "step": 342
+    },
+    {
+      "epoch": 0.9146666666666666,
+      "grad_norm": 11.416552543640137,
+      "learning_rate": 8.91891891891892e-07,
+      "loss": 1.5351,
+      "step": 343
+    },
+    {
+      "epoch": 0.9173333333333333,
+      "grad_norm": 10.296736717224121,
+      "learning_rate": 8.64864864864865e-07,
+      "loss": 1.4581,
+      "step": 344
+    },
+    {
+      "epoch": 0.92,
+      "grad_norm": 10.888872146606445,
+      "learning_rate": 8.37837837837838e-07,
+      "loss": 1.4664,
+      "step": 345
+    },
+    {
+      "epoch": 0.9226666666666666,
+      "grad_norm": 12.15527057647705,
+      "learning_rate": 8.108108108108109e-07,
+      "loss": 1.4899,
+      "step": 346
+    },
+    {
+      "epoch": 0.9253333333333333,
+      "grad_norm": 12.206395149230957,
+      "learning_rate": 7.837837837837839e-07,
+      "loss": 1.487,
+      "step": 347
+    },
+    {
+      "epoch": 0.928,
+      "grad_norm": 11.60653018951416,
+      "learning_rate": 7.567567567567569e-07,
+      "loss": 1.3755,
+      "step": 348
+    },
+    {
+      "epoch": 0.9306666666666666,
+      "grad_norm": 11.051371574401855,
+      "learning_rate": 7.297297297297298e-07,
+      "loss": 1.4968,
+      "step": 349
+    },
+    {
+      "epoch": 0.9333333333333333,
+      "grad_norm": 10.283153533935547,
+      "learning_rate": 7.027027027027028e-07,
+      "loss": 1.6037,
+      "step": 350
+    },
+    {
+      "epoch": 0.9333333333333333,
+      "eval_loss": 1.5012240409851074,
+      "eval_runtime": 1.4483,
+      "eval_samples_per_second": 20.715,
+      "eval_steps_per_second": 10.357,
+      "step": 350
+    },
+    {
+      "epoch": 0.936,
+      "grad_norm": 10.748376846313477,
+      "learning_rate": 6.756756756756758e-07,
+      "loss": 1.4126,
+      "step": 351
+    },
+    {
+      "epoch": 0.9386666666666666,
+      "grad_norm": 10.478782653808594,
+      "learning_rate": 6.486486486486487e-07,
+      "loss": 1.5083,
+      "step": 352
+    },
+    {
+      "epoch": 0.9413333333333334,
+      "grad_norm": 12.424768447875977,
+      "learning_rate": 6.216216216216217e-07,
+      "loss": 1.4426,
+      "step": 353
+    },
+    {
+      "epoch": 0.944,
+      "grad_norm": 9.786967277526855,
+      "learning_rate": 5.945945945945947e-07,
+      "loss": 1.5219,
+      "step": 354
+    },
+    {
+      "epoch": 0.9466666666666667,
+      "grad_norm": 11.543196678161621,
+      "learning_rate": 5.675675675675676e-07,
+      "loss": 1.4902,
+      "step": 355
+    },
+    {
+      "epoch": 0.9493333333333334,
+      "grad_norm": 9.170177459716797,
+      "learning_rate": 5.405405405405406e-07,
+      "loss": 1.4658,
+      "step": 356
+    },
+    {
+      "epoch": 0.952,
+      "grad_norm": 11.608868598937988,
+      "learning_rate": 5.135135135135135e-07,
+      "loss": 1.5086,
+      "step": 357
+    },
+    {
+      "epoch": 0.9546666666666667,
+      "grad_norm": 10.452662467956543,
+      "learning_rate": 4.864864864864865e-07,
+      "loss": 1.4742,
+      "step": 358
+    },
+    {
+      "epoch": 0.9573333333333334,
+      "grad_norm": 11.031309127807617,
+      "learning_rate": 4.5945945945945953e-07,
+      "loss": 1.4892,
+      "step": 359
+    },
+    {
+      "epoch": 0.96,
+      "grad_norm": 13.233342170715332,
+      "learning_rate": 4.324324324324325e-07,
+      "loss": 1.4468,
+      "step": 360
+    },
+    {
+      "epoch": 0.96,
+      "eval_loss": 1.5004833936691284,
+      "eval_runtime": 1.4505,
+      "eval_samples_per_second": 20.682,
+      "eval_steps_per_second": 10.341,
+      "step": 360
+    },
+    {
+      "epoch": 0.9626666666666667,
+      "grad_norm": 10.92369556427002,
+      "learning_rate": 4.0540540540540546e-07,
+      "loss": 1.4224,
+      "step": 361
+    },
+    {
+      "epoch": 0.9653333333333334,
+      "grad_norm": 10.605429649353027,
+      "learning_rate": 3.7837837837837843e-07,
+      "loss": 1.6563,
+      "step": 362
+    },
+    {
+      "epoch": 0.968,
+      "grad_norm": 10.552976608276367,
+      "learning_rate": 3.513513513513514e-07,
+      "loss": 1.4258,
+      "step": 363
+    },
+    {
+      "epoch": 0.9706666666666667,
+      "grad_norm": 11.395950317382812,
+      "learning_rate": 3.2432432432432436e-07,
+      "loss": 1.5738,
+      "step": 364
+    },
+    {
+      "epoch": 0.9733333333333334,
+      "grad_norm": 10.793330192565918,
+      "learning_rate": 2.972972972972973e-07,
+      "loss": 1.5261,
+      "step": 365
+    },
+    {
+      "epoch": 0.976,
+      "grad_norm": 14.17935562133789,
+      "learning_rate": 2.702702702702703e-07,
+      "loss": 1.3554,
+      "step": 366
+    },
+    {
+      "epoch": 0.9786666666666667,
+      "grad_norm": 10.378753662109375,
+      "learning_rate": 2.4324324324324326e-07,
+      "loss": 1.5517,
+      "step": 367
+    },
+    {
+      "epoch": 0.9813333333333333,
+      "grad_norm": 12.07553768157959,
+      "learning_rate": 2.1621621621621625e-07,
+      "loss": 1.488,
+      "step": 368
+    },
+    {
+      "epoch": 0.984,
+      "grad_norm": 10.496935844421387,
+      "learning_rate": 1.8918918918918921e-07,
+      "loss": 1.3883,
+      "step": 369
+    },
+    {
+      "epoch": 0.9866666666666667,
+      "grad_norm": 11.443282127380371,
+      "learning_rate": 1.6216216216216218e-07,
+      "loss": 1.4621,
+      "step": 370
+    },
+    {
+      "epoch": 0.9866666666666667,
+      "eval_loss": 1.5001485347747803,
+      "eval_runtime": 1.446,
+      "eval_samples_per_second": 20.747,
+      "eval_steps_per_second": 10.373,
+      "step": 370
+    },
+    {
+      "epoch": 0.9893333333333333,
+      "grad_norm": 13.134344100952148,
+      "learning_rate": 1.3513513513513515e-07,
+      "loss": 1.5585,
+      "step": 371
+    },
+    {
+      "epoch": 0.992,
+      "grad_norm": 11.195235252380371,
+      "learning_rate": 1.0810810810810812e-07,
+      "loss": 1.4204,
+      "step": 372
+    },
+    {
+      "epoch": 0.9946666666666667,
+      "grad_norm": 11.911666870117188,
+      "learning_rate": 8.108108108108109e-08,
+      "loss": 1.5624,
+      "step": 373
+    },
+    {
+      "epoch": 0.9973333333333333,
+      "grad_norm": 11.100564956665039,
+      "learning_rate": 5.405405405405406e-08,
+      "loss": 1.5177,
+      "step": 374
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 12.218923568725586,
+      "learning_rate": 2.702702702702703e-08,
+      "loss": 1.6249,
+      "step": 375
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 375,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 10,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 8.131727356170624e+16,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-375/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d9be6ec0ef23215e7a2981248f495d079f207788f8c83f8ffb26b9d7f76671d5
+size 6097